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by 
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Abstract 

Evolutionary  pressures  have  led  to  some  astonishing  eamouflage  strategies  in  the  animal 
kingdom.  Cephalopods  like  cuttlefish  and  octopus  mastered  a  rather  unique  skill;  they 
can  rapidly  adapt  the  way  their  skin  looks  in  color,  texture  and  pattern,  blending  in  with 
their  backgrounds.  Showing  a  general  resemblance  to  a  visual  background  is  one  of  the 
many  camouflage  strategies  used  in  nature.  Eor  animals  like  cuttlefish  that  can 
dynamically  change  the  way  they  look,  we  would  like  to  be  able  to  determine  which 
camouflage  strategy  a  given  pattern  serves.  Eor  example,  does  an  inexact  match  to  a 
particular  background  mean  the  animal  has  physiological  limitations  to  the  patterns  it  can 
show,  or  is  it  employing  a  different  camouflage  strategy  (e.g.,  disrupting  its  outline)? 

This  thesis  uses  a  computational  and  data-driven  approach  to  quantify  camouflage 
patterns  of  cuttlefish  in  terms  of  color  and  pattern.  Eirst,  we  assess  the  color  match  of 
cuttlefish  to  the  features  in  its  natural  background  in  the  eyes  of  its  predators.  Then,  we 
study  overall  body  patterns  to  discover  relationships  and  limitations  between  chromatic 
components.  To  facilitate  repeatability  of  our  work  by  others,  we  also  explore  ways  for 
unbiased  data  acquisition  using  consumer  cameras  and  conventional  spectrometers, 
which  are  optical  imaging  instruments  most  commonly  used  in  studies  of  animal 
coloration  and  camouflage.  This  thesis  makes  the  following  contributions:  (1)  Proposes  a 
methodology  for  scene-specific  color  calibration  for  the  use  of  RGB  cameras  for  accurate 
and  consistent  data  acquisition.  (2)  Introduces  an  equation  relating  the  numerical  aperture 
and  diameter  of  the  optical  fiber  of  a  spectrometer  to  measurement  distance  and  angle, 
quantifying  the  degree  of  spectral  contamination.  (3)  Presents  the  first  study  assessing  the 
color  match  of  cuttlefish  {S.  officinalis)  to  its  background  using  in  situ  spectrometry.  (4) 
Develops  a  computational  approach  to  pattern  quantification  using  techniques  from 
computer  vision,  image  processing,  statistics  and  pattern  recognition;  and  introduces 
Cuttlefish? 2x5,  the  first  database  of  calibrated  raw  (linear)  images  of  cuttlefish. 

Thesis  Supervisor:  Ruth  Rosenholtz 
Title:  Principal  Research  Scientist 
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Chapter  1 

Introduction 

Evolutionary  pressures  have  led  to  some  astonishing  eamouflage  strategies  in  the 
animal  kingdom.  One  kind  of  spittlebug  evolved  to  look  like  bird  droppings.  Some  fish  in 
Amazonian  waters  adopted  the  appearanee  of  fallen  leaves,  and  spend  all  daylight  hours 
motionless  on  the  riverbed  (Stevens  and  Merilaita  2009).  In  the  tropies,  brightly  eolored 
wing  patterns  of  many  butterflies  send  warning  signals  to  predators  indieating  that  these 
prey  are  toxie  and  should  not  be  eonsumed.  Some  of  these  butterflies  are  perfeetly 
palatable;  they  are  only  mimieking  the  appearanee  of  their  toxie  cousins  who  live  nearby 
(Boggs  et  al.  2003).  Cephalopods  like  cuttlefish  and  octopus,  which  lost  their  hard  outer 
shell  millions  of  years  ago,  now  depend  on  camouflage  for  survival,  and  have  mastered  a 
rather  unique  skill:  they  can  rapidly  adapt  the  way  their  skin  looks  in  color,  texture  and 
pattern,  blending  in  with  their  visual  backgrounds  (Hanlon  and  Messenger  1996; 
Messenger  2001b)  -  a  trait  noted  in  Aristotle’s  Historia  Animalium  (Aristotle  1910),  and 
grossly  exaggerated  in  many  Greek  myths.  But  it  is  not  only  their  ability  to  dynamically 
change  their  looks  that  makes  cephalopod 
camouflage  impressive;  they  match  their 
surroundings  in  color  and  pattern  without 
actually  sensing  color  with  their  eyes 
(Mathger  et  al.  2006;  Marshall  and 
Messenger  1996).  Figure  1.1  shows  a 
small  subset  of  the  many  different  body 
patterns  of  cuttlefish  we  observed  during 
the  field  season  of  201 1  in  Urla,  Turkey. 

One  immediately  wonders  their 
capabilities:  “How  many  different 
camouflage  patterns  can  cuttlefish  show?” 

“How  well  does  one  pattern  camouflage 
the  cuttlefish  compared  to  another  it  could 
have  shown?”  “Does  each  of  these 
patterns  fool  a  different  aspect  of  the 
predator’s  visual  system?”  These  are 
arguably  the  most  important  questions  in 
the  study  of  biological  camouflage  today, 
generalizing  to  animals  other  than 
cuttlefish.  We  can  best  investigate  such 
questions  using  a  comprehensive  approach 
that  takes  into  account  the  properties  of  the 
natural  environment  camouflage  is 
displayed  against,  the  ambient  light  field 
and  the  visual  system  of  the  observer(s)  viewing  the  camouflage. 

To  interpret  a  scene  with  a  camouflaged  animal— in  a  laboratory  setting  or  in  the  wild- 
-  we  would  like  to  be  able  to  describe  the  animal’s  camouflage  quantitatively,  so  we  can 
compare  certain  properties  of  its  pattern  to  those  of  its  environment.  For  example,  we 
would  like  to  be  able  to  determine  how  closely  the  colors  of  the  animal’s  body  pattern 


What  is  camouflage? 

There  are  many  ways  to  camouflage  without 
having  a  body  pattern  that  exactly  matches,  or 
resembles  a  particular  background.  In  biology, 
camouflage  is  an  umbrella  term  that  describes 
all  strategies  used  for  concealment,  including 
body  patterning  and  coloration,  mimicking 
behavior,  and  motion  (Stevens  and  Merilaita 
2009).  While  there  is  no  widespread  consensus 
on  the  definitions  of  camouflage  in  the  biology 
community,  they  are  often  described  based  on 
their  function  or  strategy,  i.e.  the  function  of 
the  evolutionary  adaptation,  such  as  breaking 
up  form;  as  opposed  to  the  strategy,  the  specific 
perceptual  processes  targeted,  e.g.  does  the 
pattern  disrapt  normal  edge  detection 
mechanisms?  In  this  thesis,  we  are  most 
interested  in  the  crypsis  function,  which  aims  to 
initially  prevent  detection  and  includes 
strategies  of  background  matching,  distractive 
markings,  disraptive  coloration,  self-shadow 
concealment,  obliterative  shading  and  flicker- 
fusion  camouflage.  Figure  1.2  shows  examples 
of  camouflage  strategies  associated  with  the 
crypsis  function. 
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match  the  colors  of  its  background.  If  we  have  a  model  representing  the  visual  system  of 
a  predator,  we  might  even  be  able  to  assess  whether  the  animal’s  colors  are 
distinguishable  from  those  of  its  baekground  in  the  eyes  of  that  predator.  In  the  same 
way,  we  can  investigate  whether  there  are  any  similarities  between  the  spatial 
composition  of  a  pattern  (e.g.,  lines,  splotches,  dots,  ete.)  and  the  features  found  in  its 
natural  baekground.  Two  pre-eonditions  must  be  met  before  undertaking  such 
quantitative  analyses.  First,  the  data  representing  the  camouflage  scene  must  be  aequired 
in  an  unbiased  fashion,  using  the  appropriate  ealibrated  instrument(s).  In  the  field  of 
animal  eoloration  and  eamouflage,  most  commonly  used  instruments  for  data  aequisition 
are  consumer  eameras,  conventional  spectrometers  and  multi/hyper-spectral  imagers. 
Seeond,  the  methodology  used  for  pattern  quantifieation  must  be  objective,  free  of 
subjective  input  from  human  observers,  to  facilitate  repeatability  of  work  by  other 
researehers. 

In  this  thesis,  we  use  a  data-driven  and  computational  approach  to  quantify 
camouflage  patterns  of  European  euttlefish  {Sepia  officinalis)  using  techniques  from 
computer  vision,  image  proeessing,  statistics  and  pattern  recognition.  For  cuttlefish  (and 
other  eephalopods  sueh  as  squid  and  oetopus),  the  body  pattern,  describing  the  overall 
appearanee  of  the  animal  at  any  given  moment,  consists  of  chromatic  (e.g.,  light  or  dark), 
textural  (e.g.,  smooth  or  rough),  postural  (e.g.,  skin  papillae  and  arm  postures)  and 
locomotor  (e.g.,  buried  or  hovering)  components,  all  of  whieh  ean  be  combined  in  many 
ways  (Hanlon  and  Messenger  1996;  Messenger  2001b).  The  visual  nature  of  cephalopod 
communication  and  their  versatility  have  led  to  thinking  of  the  produetion  of  body 
patterns  as  behaviors  (Paekard  and  Hoehberg  1977)  mueh  like  feeding,  foraging, 
reproduction  and  communication.  We  utilize  images  of  camouflage  scenes  collected  with 
both  consumer  eameras  and  conventional  speetrometers.  To  ensure  our  analysis  of 
eamouflage  meets  the  necessary  pre-conditions  mentioned  above,  we  first  explore  ways 
of  aequiring  unbiased  data  using  commereial-off-the-shelf  cameras  (Chapter  2)  and 
conventional  speetrometers  (Chapter  3).  Then,  we  investigate  biologically  important 
questions  regarding  cuttlefish  eamouflage.  In  Chapter  4,  we  present  the  first  study  that 
assesses  color  match  of  cuttlefish  in  the  eyes  of  its  predators,  using  spectral  data  collected 
in  situ.  Quantifying  the  eolor  matehing  abilities  of  cuttlefish  is  important  because 
euttlefish  are  eolorblind  (Marshall  and  Messenger  1996;  Mathger  et  al.  2006).  In 
addition,  the  properties  of  ambient  light  that  penetrates  to  euttlefish  habitats  underwater 
generally  makes  all  objects  appear  blue  or  green,  except  for  those  at  very  shallow  depths 
(Ahlen  2005;  Jerlov  1976;  Akkaynak  et  al.  2011;  Smith  and  Baker  1978;  Vasilescu 
2009).  What  degree  of  eolor  mateh  is  sufficient  to  fool  the  visual  systems  of  predators  in 
sueh  chromatically  limited  underwater  environments? 

After  investigating  color  match,  we  move  on  to  the  analysis  of  entire  body  patterns  of 
cuttlefish.  This  is  a  difficult  problem  because  models  of  visual  pattern  perception  for  fish 
predators  do  not  exist;  and  therefore  we  ean  only  approaeh  this  problem  from  the  point  of 
view  of  the  human  visual  system.  Sinee  cuttlefish  ean  dynamically  and  rapidly  change  the 
way  their  body  pattern  looks,  it  is  not  always  straightforward  to  determine  which 
camouflage  strategy  their  pattern  might  serve.  If  they  do  not  display  a  good  resemblance 
to  their  baekgrounds,  is  it  because  they  are  using  a  different  eamouflage  strategy  (e.g., 
disruptive  eoloration,  see  Figure  1.2),  or  beeause  they  have  physiologieal  limitations  that 
prevent  them  from  generating  a  pattern  with  a  good  match  to  that  background?  We 
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investigate  the  camouflage  pattern  generating  capabilities  of  cuttlefish  from  calibrated 
photographs  in  Chapter  5 .  Understanding  limitations  to  the  patterns  cuttlefish  show  can 
give  us  insights  into  determining  whether  cuttlefish  are  capable  of  employing  more  than 
one  camouflage  strategy.  Even  though  it  has  not  been  experimentally  proven  for 
cephalopods,  Hanlon  and  colleagues  present  evidence  of  disruptive  coloration  in  two 
different  species  of  cuttlefish  (Hanlon  et  al.  2009;  Hanlon  et  al.  2013).  If  cuttlefish  can 
indeed  employ  different  camouflage  strategies,  when  we  investigate  the  structure  of  the 
space  their  patterns  form,  we  might  expect  to  observe  well-separated,  discrete  number  of 
clusters,  instead  of  a  continuum.  Such  analyses  can  contribute  to  the  ongoing  research 
regarding  how  many  camouflage  patterns  cuttlefish  can  show,  and  in  turn,  help  answer 
how  many  total  camouflage  patterns  there  may  be  in  all  of  animal  kingdom. 

Imaging  camouflage  patterns 

The  tools  used  to  study  camouflage,  particularly  to  image  animals  and  their 
backgrounds  are  important  because  the  appearance  of  a  camouflage  scene  may  be 
different  when  viewed  by  a  non-human  animal’s  visual  system.  Indeed,  our  lack  of 
consideration  or  knowledge  of  the  visual  mechanisms  of  the  relevant  observers  is  thought 
to  be  one  of  the  major  obstacles  hindering  full  understanding  of  camouflage  strategies 
(Stevens  and  Merilaita  2009).  We  know  less  about  the  visual  systems  of  animals  than  we 
do  about  humans’  (Stevens  et  al.  2007;  Mathger  et  al.  2008),  but  taking  into  account  the 
visual  capabilities  of  the  observer(s)  viewing  a  camouflage  pattern  is  important  because 
analyzing  color  signals  from  the  point-of-view  of  humans  and  making  inferences  about 
their  appearances  to  animals  often  produces  erroneous  conclusions.  This  is  because 
animal  visual  systems  differ  from  ours  in  important  ways.  For  example,  birds  have  four 
photoreceptors,  one  more  than  humans  with  normal  color  vision.  In  addition,  they  are 
sensitive  to  the  ultraviolet  (UV)  part  of  the  electromagnetic  spectrum  (Hart  et  al.  1998), 
to  which  we  are  not.  In  a  recent  publication,  Stoddard  and  Stevens  (Stoddard  and  Stevens 
2011)  showed  that  some  common  cuckoo  eggs  (which  are  laid  in  nests  of  other  species 
and  often  hatched  and  raised  by  unsuspecting  host  birds)  appeared  to  have  a  good  color 
match  to  the  host  eggs  when  viewed  by  humans,  but  had  clear  and  quantifiable 
differences  when  modeled  in  avian  color  space.  Their  study  highlights  a  key  point:  the 
optical  instrument  that  will  be  used  to  image  camouflage  pattern  must  be  capable  of 
recording  information  in  the  parts  of  the  electromagnetic  spectrum  to  which  relevant 
observers  are  sensitive. 

Three  kinds  of  optical  instruments  are  used  for  imaging  animal  patterns: 
consumer  cameras  (also  known  as  RGB  cameras),  spectrometers  and  hyper-spectral 
imagers  (Akkaynak  2014).  Among  these,  hyper-spectral  imagers  provide  the  most 
comprehensive  data  because  in  addition  to  recording  a  spatial  image,  they  also  densely 
sample  the  electromagnetic  spectrum,  creating  an  image  of  dimensions  N  xMx  P.  These 
dimensions  correspond  to  height  (N),  width  (M)  and  the  number  of  spectral  samples  (P, 
typically  between  70-100).  Despite  being  ideal  instruments  for  imaging  animal  patterns  in 
color,  hyper-spectral  imagers  are  least  commonly  used  for  animal  coloration  studies 
because  of  their  high  cost  and  physical  bulkiness.  In  addition,  their  slow  imaging  times 
that  make  it  difficult  to  photograph  moving  objects  like  animals  in  the  wild.  Consumer 
cameras,  on  the  other  hand,  provide  a  compact  and  affordable  alternative  to  hyper- 
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spectral  imagers  because  they  also  do  spatial  imaging;  however  they  are  limited  in  color 
capture  capabilities. 


Figure  1.1  The  versatility  of  cuttlefish,  and  its  remarkahle  ability  to  comhine  pigments  with 
textural  elements  and  body  postures  makes  these  animals  perfect  models  for  the  study  of 
camouflage.  Here  we  show  a  subset  of  body  patterns,  used  for  communication  and 
camouflage,  that  we  observed  in  cuttlefish’s  (S.officinalis)  natural  habitat  on  the  Aegean 
coast  of  Turkey.  Image  credits:  Derya  Akkaynak,  Justine  J.  Allen  &  Roger  T.  Hanlon. 
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By  design,  consumer  cameras  sample  the  electromagnetic  spectrum  narrowly, 
creating  images  of  dimensions  N  x  Mx  3,  and  the  parts  of  the  spectrum  they  record  data 
from  are  chosen  to  overlap  with  the  sensitivities  of  the  human  visual  system  (Akkaynak 
et  al.  2014;  Stevens  et  al.  2007).  Even  when  it  is  desired  or  acceptable  to  capture  a 
photograph  just  in  the  visible  range,  there  are  a  number  of  manual  processing  steps 
required  before  camera-dependent  colors  can  be  transformed  into  a  camera-independent 
color  space,  which  is  a  minimum  requirement  to  facilitate  repeatability  of  work  by  others 
in  the  context  of  scientific  applications.  In  the  field  of  animal  coloration  and  camouflage, 
the  limitations  of  consumer  cameras  and  the  need  to  manually  process  their  images  were 
generally  overlooked  until  the  work  of  Stevens  et  al.  (2007),  who  emphasized  that  in¬ 
camera  processed  photographs  (e.g.,  images  in  jpg  format)  may  contain  artifacts  that 
cannot  be  removed,  compromising  data  quality.  In  this  work,  we  expand  their  study  that 
focused  on  consistency  of  colors  captured  with  consumer  cameras,  and  introduce  a 
method  to  also  increase  the  accuracy  of  color  capture  using  consumer  cameras.  We  call 
this  method  scene-specific  color  calibration,  and  describe  it  in  detail  in  Chapter  2. 

To  overcome  the  color  limitations  of  consumer  cameras,  conventional 
spectrometers  with  optical  fibers  are  frequently  used  in  their  place,  or  in  tandem,  to 
measure  colors  from  an  animal’s  habitat,  nest,  eggs,  body  parts,  skin,  fur  and  plumage 
(Akkaynak  2014).  Spectrometers  sample  the  electromagnetic  spectrum  densely  with  the 
capability  to  record  information  outside  of  the  visible  range.  However,  the  image  they 
record  is  a  single  pixel  big  (i.e.,  has  dimensions  1  x  1  x  P),  requiring  multiple 
measurements  on  a  carefully  gridded  setup,  and  a  complex  reconstruction  scheme  to 
capture  spectra  for  the  entire  layout  of  a  pattern.  In  addition,  depending  on  the  optical 
properties  of  a  fiber,  it  may  not  be  possible  to  resolve  spectra  from  features  that  are  very 
small  without  colors  from  neighboring  features  mixing  in.  For  features  that  are  large 
enough,  recording  uncontaminated  spectra  demands  the  positioning  of  the  optical  fiber  as 
close  as  possible  to  surface  being  measured,  without  touching  it.  This  is  standard  practice 
in  laboratory  setups  but  not  easily  accomplished  in  field  studies  because  it  is  challenging 
to  get  the  fiber  very  close  to  freely  behaving  animals  in  the  wild.  In  Chapter  3,  we 
investigate  what  kinds  of  camouflage  assessment  errors  contaminated  spectra  can  lead  to 
if  the  surveyor  fails  to  get  very  close  to  the  feature  being  measured. 

Quantifying  camouflage  patterns 

Given  a  camouflage  pattern,  we  would  like  to  be  able  to  determine  which  aspects 
of  the  visual  background  it  may  match,  or  which  camouflage  strategy  it  serves.  This  is 
not  straightforward  because  quantitative  models  describing  camouflage  strategies  do  not 
exist.  Mathger  and  colleagues  (2008)  were  to  first  to  quantitatively  analyze  the  color 
matching  abilities  of  cuttlefish.  They  measured  reflectance  spectra  from  ten  points  on 
animals  and  background  substrates  in  the  laboratory  and  used  normalized  Euclidean 
distance  to  quantify  spectral  difference.  They  found  that  cuttlefish  showed  a  better 
spectral  match  to  natural  substrates  than  they  did  for  artificial  substrates,  but  did  not 
analyze  the  discriminability  of  colors  according  to  the  visual  systems  of  predators.  Chiao 
and  colleagues  (2011)  also  performed  a  color  match  assessment  in  the  laboratory,  but 
they  used  a  hyper-spectral  imager  to  capture  body  patterns.  They  modeled  the  visual 
systems  of  hypothetical  predators  and  found  that  the  colors  cuttlefish  produced  were 
indistinguishable  in  the  eyes  of  predators.  In  both  studies,  the  natural  substrates  used 
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were  sands  and  gravel.  Cuttlefish  live  in  habitats  ranging  from  the  Mediterranean  to  the 
eoral  reefs  of  West  Africa,  which  have  a  diversity  of  background  substrates  that  extend 
beyond  sands  and  gravel  including  three-dimensional  elements  like  sea  grass,  pinna, 
rocks,  boulders,  etc.  We  advanced  existing  work  by  travelling  to  one  of  the  natural 
habitats  of  cuttlefish  in  the  Aegean  Sea  and  conducting  the  first  in  situ  spectroscopic 
survey  of  color  and  luminance  from  cuttlefish  skin  and  substrates  from  that  habitat.  Then, 
we  calculated  color  and  luminance  contrast  using  the  Vorobyev-Osario  receptor  noise 
model  (Vorobyev  and  Osorio  1998),  which  allowed  us  to  assess  the  discriminability  of 
colors  in  the  eyes  of  hypothetical  di-  and  tri-chromat  predators  (Chapter  4).  In  this  field 
study,  we  focused  only  on  quantifying  spectral  discriminability,  omitting  spatial 
configurations  of  body  patterns,  which  we  investigate  in  Chapter  5. 

To  compare  one  camouflage  pattern  with  another  and  quantify  the  differences 
between  them,  it  is  important  to  use  a  dataset  that  is  free  of  instrument  bias,  and  an 
analysis  method  that  does  not  contain  subjective  judgments.  To  facilitate  repeatability  of 
our  work  by  others,  prior  to  our  analysis  of  overall  camouflage  patterns,  we  created 
Cuttlefish? 2x5,  the  first  database  consisting  of  linear  images  of  camouflaged  cuttlefish.  It 
was  important  to  establish  this  database  before  asking  biological  camouflage  questions 
because  previous  work  describing  animal  patterns  and  coloration  often  based  their 
analyses  on  images  in  jpg  format.  Such  images  are  standard  outputs  of  consumer  cameras 
and  have  been  processed  and  compressed  in  ways  that  they  no  longer  maintain  a  linear 
relationship  to  scene  radiance  (Stevens  et  al.  2007;  Akkaynak  et  al.  2014).  This  means 
that  they  may  not  accurately  represent  the  pattern  that  was  photographed.  In  addition, 
earlier  studies  that  describe  camouflage  patterns  involved  subjective  judgments  by  human 
observers.  While  assessments  of  patterns  by  human  observers  is  quick  and  does  not 
require  any  specialized  instruments,  it  is  inherently  subjective  and  the  resulting 
qualitative  descriptions  often  vary  between  observers,  causing  low  repeatability  (Stevens 
2011).  Thus,  camera  bias  and  processing  artifacts  were  minimized  for  the  images  in  our 
database,  facilitating  repeatability  of  our  results  by  other  researchers.  Based  on  the 
images  in  the  Cuttlefish? 2x5  database,  we  focused  on  three  aspects  of  cuttlefish 
camouflage  patterns  and  used  computational  methods  to  investigate  them:  the 
relationship  between  the  level  of  expression  of  components  in  terms  of  intensity  and 
mottleness;  the  structure  of  the  camouflage  pattern  space  (e.g.,  do  patterns  form  discrete 
clusters  or  a  continuum),  and  whether  cuttlefish  can  show  all  of  the  patterns  they  are 
theoretically  capable  of  producing.  These  questions  are  investigated  in  detail  in  Chapter 
5. 
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camouflage,  meaning  all  strategies  involved  in  concealment,  including  prevention  of  detection  and 
recognition 

crypsis  (initially  preventing  detection): 

(a)  background  matching,  where  the  appearance  generally  matches  the  color,  lightness,  and 
pattern  of  one  (specialist),  or  several  (compromise)  background  types 

(b)  self-shadow  concealment,  where  directional  light  (which  would  leave  to  the  creation  of 
shadows)  is  cancelled  out  by  countershading. 

(c)  obliterative  shading,  where  countershading  leads  to  the  obliteration  of  three-dimensional 
form 

(d)  disruptive  coloration,  being  a  set  of  markings  that  creates  the  appearance  of  false  edges 
and  boundaries,  and  hinders  the  detection  or  recognition  of  an  objet’s  or  part  of  an  object’s  true 
outline  and  shape 

(e)  flicker-fusion  camouflage,  where  markings  such  as  stripes  blur  during  motion  to  match  the 
color/lightness  of  the  general  background,  preventing  detection  of  the  animal  when  in  motion 

(f)  distractive  markings,  which  direct  the  attention  or  gaze  of  the  receiver  from  traits  that 
would  give  away  the  animal  (such  as  the  outline) 

masquerade,  where  recognition  is  prevented  by  resembling  an  iminteresting  object,  such  as  a  leaf 
or  a  stick 


motion  dazzle,  where  markings  make  estimates  of  speed  and  trajectory  difficult  by  an  observer 
motion  camouflage,  movement  in  a  fashion  that  decreases  the  probability  of  movement  detection 


(d)  (e)  (f) 

Figure  1.2  While  there  is  no  widespread  consensus  on  the  definitions  of  biological  camouflage,  Stevens 
and  Merilaita  (2009)  provide  the  most  comprehensive  set  of  descriptions,  which  we  adopt  in  this 
work,  (a)  Background  matching  by  cuttlefish  (b)  Counter-shaded  grey  reef  shark  (c)  Obliterative 
shading  makes  one  of  the  two  decoy  ducks  in  this  photograph  disappear  (on  the  right,  practically 
invisible)  while  the  non-countershaded  on  the  left  is  visible  (d)  Disruptive  pattern  of  a  panda  bear  (e) 
The  patterning  of  the  common  European  viper  might  trigger  a  flicker-fusion  illusion  (f)  False  eye 
spots  serve  as  distractive  markings  on  this  brush-footed  butterfly.  Image  credits:  (a):  Hanlon  Lab, 
(b)-(f):  Wikipedia. 


Chapter  2 


Unbiased  data  acquisition:  Commerciai-off-the-sheif  digitai 
cameras^ 

2.1  Introduction 

State-of-the-art  hardware,  built-in  photo  enhaneement  software,  waterproof 
housings  and  affordable  prices  enable  widespread  use  of  commercial  off-the-shelf 
(COTS)  digital  cameras  in  research  laboratories.  However  it  is  often  overlooked  that 
these  cameras  are  not  optimized  for  accurate  color  capture,  but  rather  for  producing 
photographs  that  will  appear  pleasing  to  the  human  eye  when  viewed  on  small  gamut  and 
low  dynamic  range  consumer  devices  (Chakrabarti  et  al.  2009;  Scon  Joo  et  al.  2012).  As 
such,  use  of  cameras  as  black-box  systems  for  scientific  data  acquisition,  without 
knowledge  and  control  of  how  photographs  are  manipulated  inside,  may  compromise 
data  quality  and  in  turn,  hinder  repeatability.  For  example,  a  common  color-sensitive  use 
of  consumer  cameras  is  in  the  field  of  animal  coloration  and  camouflage  (Stevens  et  al. 
2007;  Pike  2011b).  How  well  do  the  colors  from  the  skin  of  a  camouflaged  animal  match 
the  colors  in  a  particular  background?  For  an  objective  analysis  of  this  question,  the 
photograph  of  the  animal  taken  by  a  researcher  using  a  certain  make  and  model  COTS 
camera  should  be  reproducible  by  another  researcher  using  a  different  camera.  This  is  not 
straightforward  to  accomplish  because  each  camera  records  colors  differently,  based  on 
the  material  properties  of  its  sensor,  and  built-in  images  processing  algorithms  are  often 
make  and  model  specific,  and  proprietary.  Thus,  for  scientific  research,  manual 
processing  of  COTS  camera  images  to  obtain  device-independent  photographs  is  crucial. 
In  this  chapter,  we  explore  the  following  question;  what  are  the  pre-  and  post-image 
acquisition  steps  necessary  so  color  capture  using  COTS  cameras  is  consistent  and 
accurate  enough  to  yield  photographs  that  can  be  used  as  scientific  data?  Our  goal  is  to 
identify  the  limitations  of  COTS  cameras  for  color  imaging  and  streamline  the  manual 
processing  steps  necessary  for  their  use  as  consistent  and  accurate  scientific  data 
acquisition  instruments. 

A  consumer  camera 
photograph  is  considered 
unbiased  if  it  has  a  known 
relationship  to  scene 
radiance.  This  can  be  a 
purely  linear  relationship, 
or  a  non-linear  one  where 
the  non-linearities  are 
precisely  known  and  can  be 
inverted.  A  linear  relationship  to  scene  radiance  makes  it  possible  to  obtain  device¬ 
independent  photographs  that  can  be  quantitatively  compared  with  no  knowledge  of  the 
original  imaging  system.  Raw  photographs  recorded  by  many  cameras  have  this  desired 
property  (Chakrabarti  et  al.  2009),  whereas  camera-processed  images,  most  commonly 


^  Parts  of  this  chapter  were  previously  published.  See  (Akkaynak  et  al.  2014)  for  full 
reference. 
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images  in  jpg  format,  do  not.  For  scientific  research,  obtaining  device-independent 
photographs  is  crucial  because  each  consumer  camera  records  colors  differently.  Only 
after  camera-dependent  colors  are  transformed  to  a  standard,  camera-independent  color 
space,  researchers  using  different  makes  and/or  models  of  cameras  can  reproduce  them. 

When  pixel  intensities  in  photographs  are  to  be  used  as  scientific  data,  it  is 
important  to  use  raw  images,  and  manually  process  them,  controlling  each  step,  rather 
than  using  camera-processed  images.  In-camera  processing  introduces  non-linearities 
through  make  and  model-specific  and  often  proprietary  operations  that  alter  the  color, 
contrast  and  white  balance  of  images.  These  images  are  then  transformed  to  a  non-linear 
RGB  space,  and  compressed  in  an  irreversible  fashion  (Figure  2.1).  Compression,  for 
instance,  creates  artifacts  that  can  be  so  unnatural  that  they  may  be  mistaken  for  cases  of 
image  tampering  (Figure  2.2)  (Farid  2011).  As  a  consequence,  pixel  intensities  in 
consumer  camera  photographs  are  modified  such  that  they  are  no  longer  linearly  related 
to  scene  radiance.  Models  that  approximate  raw  (linear)  RGB  from  non-linear  RGB 
images  (e.g.,  sRGB)  exist,  but  at  their  current  stage  they  require  a  series  of  training 
images  taken  under  different  settings  and  light  conditions  as  well  as  ground-truth  raw 
images  (Seon  Joo  et  al.  2012). 


(a)  (b) 


Figure  2.2  (a)  An  uncompressed  image,  (b)  Artifacts  after  jpg  compression:  1)  grid-like 
pattern  along  block  boundaries  2)  blurring  due  to  quantization  3)  color  artifacts  4)  jagged 
object  boundaries.  Pboto  credit:  Dr.  Hany  Farid.  Used  with  permission. 
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2.2  Background  and  related  work 

The  limitations  and  merit  of  COTS  eameras  for  scientifie  applieations  have 
previously  been  explored,  albeit  disjointly,  in  ecology  (Levin  et  al.  2005),  environmental 
sciences  (De  La  Barrera  and  Smith  2009),  systematics  (McKay  2013),  animal  coloration 
(Stevens  et  al.  2007),  dentistry  (Wee  et  al.  2006)  and  underwater  imaging  (Ahlen  2005; 
Akkaynak  et  al.  2011).  Stevens  et  al.  (Stevens  et  al.  2007)  wrote: 

“...  most  current  applications  of  digital  photography  to  studies  of  animal 
coloration  fail  to  utilize  the  full  potential  of  the  technology;  more  commonly,  they  yield 
data  that  are  qualitative  at  best  and  uninterpretable  at  worst”. 

Our  goal  is  to  address  this  issue  and  make  COTS  cameras  accessible  to 
researchers  from  all  disciplines  as  proper  data  collection  instruments.  We  focus  on  two 
aspects  of  color  capture  using  consumer  cameras:  consistency  (Ch.  2. 3-2. 7)  and  accuracy 
(Ch.  2.8).  Calibration  of  consumer  cameras  through  imaging  photographic  calibration 
targets  (i.e.,  camera  characterization)  is  a  technique  well  known  to  recreational  and 
professional  photographers  for  obtaining  consistent  colors  under  a  given  light  source. 
Researchers  in  fields  like  image  processing,  computer  vision  and  computational 
photography  also  frequently  use  this  technique  (Szeliski  2010;  Shirley  et  al.  2009;  Pharr 
and  Humphreys  2010;  Josh!  and  Jensen  2004).  However,  in  the  field  of  animal  coloration, 
the  limitations  of  consumer  cameras  and  the  need  to  manually  calibrate  them  were 
generally  overlooked  until  the  work  of  Stevens  et  al.  (2007),  who  emphasized  that  in¬ 
camera  processed  photographs  (e.g.,  images  in  jpg  format)  may  contain  artifacts  that 
cannot  be  removed,  compromising  data  quality.  In  the  first  part  of  this  chapter,  we  review 
and  streamline  the  necessary  manual  processing  steps  described  by  Stevens  et  al.  (2007) 
and  others. 

In  Chapter  2.8,  we  introduce  the  idea  of  scene-specific  color  calibration  and  show 
that  it  improves  color  transformation  accuracy  when  a  non-ordinary  scene  is 
photographed.  We  define  an  ordinary  scene  as  one  that  has  colors  that  are  within  the 
gamut  of  a  commercially  available  color  calibration  target.  Color  transformation,  which 
we  describe  in  detail  in  Chapter  2.7,  is  done  through  a  transformation  matrix  that  maps 
device-specific  colors  to  a  standard,  device-independent  space  based  on  known, 
manufacturer-specified  chromatic  properties  of  calibration  target  patches.  This 
transformation  matrix  is  generally  computed  through  solving  a  system  of  equations  with 
using  linear  least  squares  regression.  Previous  work  that  focused  on  increasing  color 
transformation  accuracy  used  standard,  off-the-shelf  calibration  targets  and  investigated 
methods  other  than  linear  least  squares  regression  to  minimize  the  error  between  captured 
and  desired  RGB  values.  Examples  of  methods  used  include  non-linear  regression 
(Westland  and  Ripamonti  2004),  constrained  least  squares  regression  (Finlayson  and 
Drew  1997),  neural  networks  (Cheung  et  al.  2004)  and  interpolation  (Johnson  1996).  Our 
approach  differs  from  existing  work  in  that  we  use  the  simplest  mathematical  solution  to 
the  system  of  equations  that  describe  the  color  transformation  (i.e.,  linear  least  squares 
regression),  and  derive  the  desired  set  of  RGB  value  directly  from  features  in  the  scene 
instead  of  commercial  photographic  calibration  targets. 

Even  though  our  work  is  motivated  by  accurate  capture  of  colors  from 
camouflage  patterns  of  animals,  our  methodology  is  applicable  to  problems  in  all  fields  of 
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science  consumer  cameras  are  used;  we  demonstrate  this  in  Chapter  2.9  using  real 
scientific  problems.  Throughout  this  work,  “camera”  will  refer  to  “COTS  digital 
cameras”,  also  known  as  “consumer,”  “digital  still,”  “trichromatic”  or  “RGB”  cameras. 
Any  references  to  RGB  will  mean  “linear  RGB”  and  non-linear  RGB  images  or  color 
spaces  will  be  explicitly  specified  as  such. 

2.3  Color  imaging  with  COTS  cameras 

Color  vision  in 
surfaces  based  on 
their  spectral 
properties.  Normal 
human  vision  is 
trichromatic;  the 
retina  has  three  cone 
photoreceptors 
referred  to  as  short  (S, 
peak  440  nm), 
medium  (M,  peak  545 
nm)  and  long  (L,  peak 
580  nm).  Multiple 
light  stimuli  with 
different  spectral 
shapes  evoke  the 
same  response.  This 
response  is 
represented  by  three 

scalars  known  as  tri-stimulus  values,  and  stimuli  that  have  the  same  tri-stimulus  values 
create  the  same  color  perception  (Wyszecki  and  Stiles  2000).  Typical  cameras  are  also 
designed  to  be  trichromatic;  they  use  color  filter  arrays  on  their  sensors  to  filter 
broadband  light  in  the  visible  part  of  the  electromagnetic  spectrum  in  regions  humans 
perceive  as  red  (R),  green  (G)  and  blue  (B).  These  filters  are  characterized  by  their 
spectral  sensitivity  curves,  unique  to  every  make  and  model  (Figure  2.4).  This  means  that 
two  different  cameras  record  different  RGB  values  for  the  same  scene. 

Human  photoreceptor  spectral  sensitivities  are  often  modeled  by  the  color 
matching  functions  defined  for  the  2°  observer  (foveal  vision)  in  the  CIE  1931  XYZ 
color  space.  Any  color  space  that  has  a  well-documented  relationship  to  XYZ  is  called 
device-independent  (Reinhard  et  al.  2008).  Conversion  of  device-dependent  camera 
colors  to  device-independent  color  spaces  is  the  key  for  repeatability  of  work  by  others; 
we  describe  this  conversion  in  Ch.  2.7  &  2.8. 

2.4  Image  formation  principles 

The  intensity  of  light  recorded  at  a  sensor  pixel  is  a  function  of  the  light  that 
illuminates  the  object  of  interest  (irradiance.  Figure  2.5),  the  light  that  is  reflected  from 
the  object  towards  the  sensor  (radiance),  the  spectral  sensitivity  of  the  sensor  and  optics 
of  the  imaging  system; 


humans  (and  other  animals)  is  used  to  distinguish  objects  and 


Figure  2.3  The  workflow  proposed  for  processing  raw  images.  Consumer 
cameras  can  be  used  for  scientific  data  acquisition  if  images  are  captured 
in  raw  format  and  processed  manually  so  that  they  maintain  a  linear 
relationship  to  scene  radiance. 


23 


(2.1) 


Ic  =  k(Y)cosei  J,^”"^Sc(A)Li(A)F(A,e)dA. 

'^min 

Here  c  is  the  color  channel  (e.g.,  R,  G,  B),  Sc(A)  is  the  spectral  sensitivity  of  that 
channel,  Li(A)  is  the  irradiance,  F(A,  0)  is  the  bi-directional  reflectance  distribution 
function  and  and  Aj^ax  denote  the  lower  and  upper  bounds  of  the  spectrum  of 
interest,  respectively  (Szeliski  2010).  Scene  radiance  is  given  by: 

Lr(A)  =  Li(A)F  (A,  0)  cosOi  (2.2) 

where  F(A,  0)  is  dependent  on  the  incident  light  direction  as  well  as  the  camera  viewing 
angle  where  0  =  (0,,  4>i,  0j.,  4>j.).  The  function  k(Y)  depends  on  optics  and  other  imaging 
parameters  and  the  cosOj  term  accounts  for  the  changes  in  the  exposed  area  as  the  angle 
between  surface  normal  and  illumination  direction  changes.  Digital  imaging  devices  use 
different  optics  and  sensors  to  capture  scene  radiance  according  to  these  principles  (Table 
!)• 


Table  1  Comparison  of  basic  properties  of  color  imaging  devices 


Device 

Spatial 

Spectral 

Image  Size 

Cost 

Spectrometer 

X 

/ 

1  X  p 

>  $2,000 

COTS  camera 

/ 

X 

n  X  m  X  3 

>$200 

Hyperspectral 

imager 

/ 

/ 

n  X  m  X  p 

>  $20,000 

2.5  Demosaicing 

In  single-sensor  cameras  the  raw  image  is  a  two  dimensional  array  (Figure  2.6a, 
inset).  At  each  pixel,  it  contains  intensity  values  that  belong  to  one  of  R,  G  or  B  channels 
according  to  the  mosaic  layout  of  the  filter  array.  Bayer  Pattern  is  the  most  commonly 
used  mosaic.  At  each  location,  the  two  missing  intensities  are  estimated  through 
interpolation  in  a  process  called  demosaicing  (Ramanath  et  al.  2002).  The  highest  quality 
demosaicing  algorithm  available  should  be  used  regardless  of  its  computation  speed 
(Figure  2.6),  because  speed  is  only  prohibitive  when  demosaicing  is  carried  out  using  the 
limited  resources  in  a  camera,  not  when  it  is  done  by  a  computer. 


wavelength(nm)  wavelength(nm) 

Figure  2.4  Human  color  matcbing  functions  for  tbe  CIE  XYZ  color  space  for  2°  observer 
and  spectral  sensitivities  of  two  cameras;  Canon  EOS  IDs  mk  II  and  Nikon  D70. 


24 


2.6  White  balancing 

In  visual  perception  and  color  reproduction,  white  has  a  privileged  status 
(Finlayson  and  Drew  1997).  This  is  because  through  a  process  called  chromatic 
adaptation  our  visual  system  is  able  to  discount  small  changes  in  the  color  of  an 
illuminant,  effectively  causing  different  lighting  conditions  to  appear  “white”  (Reinhard 
et  al.  2008).  For  example,  a  white  slate  viewed  underwater  would  still  be  perceived  as 
white  by  a  SCUBA  diver,  even  though  the  color  of  the  ambient  light  is  likely  to  be  blue 
or  green,  as  long  as  the  diver  is  adapted  to  the  light  source.  Cameras  cannot  adapt  like 
humans,  and  therefore  cannot  discount  the  color  of  the  ambient  light.  Thus,  photographs 
must  be  white  balanced  to  appear  realistic  to  a  human  observer.  White  balancing  often 
refers  to  two  concepts  that  are  related  but  not  identical;  RGB  equalization  and  chromatic 
adaptation  transform  (CAT),  described  below. 

light  spectrum  reflectance 


wavelength(nm) 

(a) 


wavelength(nm) 

(b) 


radiance 
noon  daylight 


(0 


radiance 

underwater  light  at  noon 


(d) 


Figure  2.5  (a)  Irradiance  of  daylight  at  noon  (CIE  D65  illuminant)  and  noon  daylight 
on  a  sunny  day  recorded  at  3  meters  depth  in  the  Aegean  Sea.  (b)  Reflectance  spectra 
of  blue,  orange  and  red  patches  from  a  Macbetb  ColorCbecker.  Reflectance  is  tbe 
ratio  of  reflected  light  to  incoming  light  at  each  wavelength  and  it  is  a  physical 
property  of  a  surface,  unaffected  by  the  ambient  light  field,  unlike  radiance,  (c) 
Radiance  of  the  same  patches  under  noon  daylight  on  land  and  (d)  underwater. 


In  scientific  imaging,  consistent  capture  of  scenes  often  has  more  practical 
importance  than  capturing  them  with  high  perceptual  accuracy.  White  balancing  as 
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described  here  is  a  linear  operation  modifies  photos  so  they  appear  “natural”  to  us.  For 
purely  computational  applications  in  which  human  perception  does  not  play  a  role,  and 
therefore  a  natural  look  is  not  necessary,  white  balancing  can  be  done  using  RGB 
Equalization,  which  has  less  perceptual  relevance  than  CAT,  but  is  simpler  to  implement 
(see  examples  in  Ch.  2.9).  Here,  we  describe  both  methods  of  white  balancing  and  leave 
it  up  to  the  reader  to  decide  which  method  to  use. 


2.6.1  Chromatic  Adaptation  Transform  (CAT) 

Also  called  white  point  conversion,  CAT  models  approximate  the  chromatic 
adaptation  phenomenon  in  humans,  and  have  the  general  form; 


^destination  \.^a\ 


Pd/Ps  0  0 

0  Yd/Ys  0 

0  0  Pd/Ps 


[MaK 


XYZ 


(2.3) 


where  denotes  the  3  x  N  matrix  of  colors  in  XYZ  space,  whose  appearance  is  to  be 
transformed  from  the  source  illuminant  (S)  to  the  destination  illuminant  (D);  is  a  3^3 
matrix  defined  uniquely  for  the  CAT  model  and  p,  y  and  (B  represent  the  tri-stimulus 
values  in  the  cone  response  domain  and  are  computed  as  follows: 


i=S,D 


(2.4) 


Here,  WP  is  a  3^1  vector  corresponding  to  the  white  point  of  the  light  source.  The  most 
commonly  used  CAT  models  are  Von  Kries,  Bradford,  Sharp  and  CMCCAT2000.  The 
Ma  matrices  for  these  models  can  be  found  in  (Susstrunk  et  al.  2000). 


original  scene  high-quality  demosaicing  low-quality  demosaicing 


(a)  (b)  (c) 


Figure  2.6  (a)  An  original  scene.  Inset  at  lower  left:  Bayer  mosaic,  (b)  Close-ups  of 
marked  areas  after  high-quality  (adaptive)  and  (c)  after  low-quality  (non-adaptive) 
demosaicing.  Artifacts  shown  here  are  zippering  on  the  sides  of  the  ear  and  false  colors 
near  the  white  pixels  of  the  whiskers  and  the  eye. 


2.6.2  RGB  Equalization 

RGB  equalization,  often  termed  the  “wrong  von  Kries  model”  (Westland  and 
Ripamonti  2004),  effectively  ensures  that  the  R,G  and  B  values  recorded  for  a  gray 
calibration  target  are  equal  to  each  other.  For  a  pixel  p  in  the  color  channel  of  a  linear 
image,  RGB  equalization  is  performed  as: 
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i  =  R,  G,  B 


(2.5) 


pWB  = 


Pi 

WSi-DSi 


where  is  the  intensity  of  the  resulting  white  balanced  pixel  in  the  channel,  and 
DSi  and  WSi  are  the  values  of  the  dark  standard  and  the  white  standard  in  the 
channel,  respectively.  The  dark  standard  is  usually  the  black  patch  in  a  calibration  target, 
and  the  white  standard  is  a  gray  patch  with  uniform  reflectance  spectrum  (often,  the  white 
patch).  A  gray  photographic  target  (Figure  2.7)  is  an  approximation  to  a  Lambertian 
surface  (one  that  appears  equally  bright  from  any  angle  of  view)  and  has  a  uniformly 
distributed  reflectance  spectrum.  On  such  a  surface  the  RGB  values  recorded  by  a  camera 
are  expected  to  be  equal;  but  this  is  almost  never  the  case  due  to  a  combination  of  camera 
sensor  imperfections  and  spectral  properties  of  the  light  field  (Westland  and  Ripamonti 
2004);  RGB  equalization  compensates  for  that. 


ITPTffi 


Hill 

■ 

(a) 


wavelength(nm) 


(b) 


Figure  2.7  (a)  Examples  of  photographic  calihration  targets.  Top  to  bottom:  Sekonik 
Exposure  Profile  Target  II,  Digital  Kolor  Kard,  Macheth  ColorChecker  (MCC), 
ColorChecker  Digital,  (h)  Reflectance  spectra  (400-700nm)  of  SpectralouTM  targets 
(hlack  curves,  prefixed  with  SRS-),  gray  patches  of  the  MCC  (purple),  and  a  white  sheet 
of  printer  paper  (blue).  Note  that  MCC  23  has  a  flatter  spectrum  than  the  white  patch 
(MCC  19).  The  printer  paper  is  bright  and  reflects  most  of  the  fight,  but  it  does  not  do  so 
uniformly  at  each  wavelength. 


2.7  Color  transformation 

Two  different  cameras  record  different  RGB  values  for  the  same  scene  due  to 
differences  in  color  sensitivity.  This  is  true  even  for  cameras  of  the  same  make  and  model 
(Stevens  et  al.  2007).  Thus,  the  goal  of  applying  a  color  transformation  is  to  minimize 
this  difference  by  converting  device-specific  colors  to  a  standard,  device-independent 
space  (Figure  2.8).  Such  color  transformations  are  constructed  by  imaging  calibration 
targets.  Standard  calibration  targets  contain  patches  of  colors  that  are  carefully  selected  to 
provide  a  basis  to  the  majority  of  natural  reflectance  spectra.  A  transformation  matrix  T 
between  camera  color  space  and  a  device-independent  color  space  is  computed  as  a  linear 
least  squares  regression  problem; 


y^XYZ  _ 

''ground truth  ^  ''linear 


(2.6) 
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Here,  truth  are  3  x  N  matrices  where  N  is  the  number  of  patches  in 

the  calibration  target.  The  ground  truth  XYZ  tri-stimulus  values  Vground  truth  can  either 
be  the  published  values  specific  to  that  chart,  or  they  could  be  calculated  from  measured 
spectra  (Ch.  2.7).  The  RGB  values  Vunear  obtained  from  the  linear  RGB  image  of  the 
calibration  target.  Note  that  the  published  XYZ  values  for  color  chart  patches  can  be  used 
only  for  the  illuminants  that  were  used  to  construct  them  (e.g.,  CIE  illuminants  D50  or 
D65);  for  other  illuminants  a  white  point  conversion  (Eq.  2.3&2.4)  should  first  be 
performed  on  linear  RGB  images. 


before  color  transformation  after  color  transformation 

(device-dependent  colors)  (device-independent  colors) 


Figure  2.8  Chromaticity  of  Macbeth  ColorChecker  patches  captured  by  two  cameras, 
whose  sensitivities  are  given  in  Fig.  5  in  device-dependent  and  independent  color 
spaces. 


The  3x3  transformation  matrix  T  (see  (Westland  and  Ripamonti  2004)  for 
other  polynomial  models)  is  then  estimated  from  Eq.  2.6: 

T  =  trutuKiLA*  (2.7) 

where  the  superscript  +  denotes  the  Moore -Penrose  pseudo-inverse  of  the  matrix 
This  transformation  T  is  then  applied  to  a  white-balanced  novel  image  lunear- 

rXYZ  _  T-'rRGB  r'j  a\ 

^corrected  ”  ^  ^linear 

to  obtain  the  color-corrected  image  I  corrected^  which  is  the  linear,  device-independent 
version  of  the  raw  camera  output.  The  resulting  image  Icorrected  needs  to  be  converted  to 
RGB  before  it  can  be  displayed  on  a  monitor.  There  are  many  RGB  spaces  and  one  that 
can  represent  as  many  colors  as  possible  should  be  preferred  for  computations  (e.g., 
Adobe  wide  gamut)  but  when  displayed,  the  image  will  eventually  be  shown  within  the 
boundaries  of  the  monitor’s  gamut. 

In  Eq.  2.6,  we  did  not  specify  the  value  of  N,  the  number  of  patches  used  to  derive 
the  matrix  T.  Commercially  available  color  targets  vary  in  the  number  of  patches  they 
have,  ranging  between  tens  and  hundreds.  In  general,  higher  number  of  patches  used  does 
not  guarantee  an  increase  in  color  transformation  accuracy.  Alsam  and  Finlayson  (Alsam 
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and  Finlayson  2008)  found  that  13  of  the  24  patches  of  a  Macbeth  ColorChecker  (MCC) 
are  sufficient  for  most  transformations.  Intuitively,  using  patches  whose  radiance  spectra 
span  the  subspace  of  those  in  the  scene  yields  the  most  accurate  transforms;  we 
demonstrate  this  in  Figure  2.10.  Given  a  scene  that  consists  of  a  photograph  of  a  MCC 
taken  under  daylight,  we  derive  T  using  an  increasing  number  of  patches  (1-24  at  a  time) 
and  compare  the  total  color  transformation  error  in  each  case.  We  use  the  same  image  of 
the  MCC  for  training  and  testing  because  this  simple  case  provides  a  lower  bound  on 
error.  We  quantify  total  error  using; 

e  =  +  (At  -  -  BarY  (2.9) 

where  an  LAB  triplet  is  the  representation  of  an  XYZ  triplet  in  the  CIE  LAB  color  space 
(which  is  perceptually  uniform);  i  indicates  each  of  the  N  patches  in  the  MCC  and  GT  is 


Figure  2.9  Using  more  patches  for  a  color  transformation  does  not  guarantee  increased 
transformation  accuracy.  In  this  example,  color  transformation  error  is  computed  after  1-24 
patches  are  used.  There  were  many  possible  ways  the  patches  could  have  been  selected,  only  three 
are  shown  here.  Regardless  of  patch  ordering,  overall  color  transformation  error  is  minimized 
after  the  inclusion  of  the  18th  patch.  First  six  patches  of  orders  1  and  2  are  chromatic  and  for  order 
3,  they  are  achromatic.  The  errors  associated  with  order  3  are  higher  initially  because  the  scene, 
which  consists  of  a  photo  of  a  Macbeth  ColorChecker,  is  mostly  chromatic.  Note  that  it  is  not 
possible  to  have  the  total  error  be  identically  zero  even  in  this  simple  example  due  to  numerical 
error  and  noise. 


the  ground-truth  value  for  the  eorresponding  pateh.  Initially,  the  total  error  depends  on 
the  ordering  of  the  eolor  patehes.  Sinee  it  would  not  be  possible  to  simulate  24!  (6.2045  x 
1023)  different  ways  the  patehes  could  be  ordered,  we  computed  error  for  three  cases  (see 
Figure  2.9  legend).  Initial  error  is  the  highest  for  patch  order  3  because  the  first  six 
patches  of  this  ordering  are  the  achromatic  and  this  transformation  does  poorly  for  the 
MCC,  which  is  composed  of  mostly  chromatic  patches.  Patch  orderings  1  and  2  on  the 
other  hand,  start  with  chromatic  patches  and  the  corresponding  initial  errors  are  roughly 
an  order  of  magnitude  lower.  Regardless  of  patch  ordering,  the  total  error  is  minimized 
after  the  inclusion  of  the  1 8th  patch. 
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2.8  Scene-specific  color  calibration  (SSCC) 

In  Ch.  2.7,  we  outlined  the  proeess  for  building  a  3  x  3  matrix  T  that  transforms 
eolors  from  a  eamera  eolor  spaee  to  the  standard  CIE  XYZ  eolor  spaee.  It  is  apparent 
from  this  process  that  the  calibration  results  are  heavily  dependent  on  the  choice  of  the 
calibration  target,  or 
the  specific  patches 
used.  Then,  we  can 
hypothesize  that  if 
we  had  a  calibration 
target  that  contained 
all  the  colors  found 
in  a  given  scene, 
and  only  those 
colors,  we  would 
obtain  a  color 

transformation  with  minimum  error.  In  other  words,  if  the  colors  used  to  derive  the 
transformation  T  were  also  the  colors  used  to  evaluate  calibration  performance,  the 
resulting  error  would  be  minimal  -  this  is  the  goal  of  scene-specific  color  calibration. 

The  color  signal  that  reaches  the  eye,  or  camera  sensor,  is  the  product  of 
reflectance  and  irradiance  (Figure  2.5),  i.e.  radiance  (Eq.  2.1&2.2).  Therefore,  how  well  a 
calibration  target  represents  a  scene  depends  on  both  the  chromatic  composition  of  the 
features  in  the  scene  (reflectance)  and  the  ambient  light  profile  (irradiance).  For  example, 
a  scene  viewed  under  daylight  will  appear  monochromatic  if  it  only  contains  different 
shades  of  a  single  hue,  even  though  daylight  is  a  broadband  light  source.  Similarly,  a 
scene  consisting  of  an  MCC  will  appear  monochromatic  when  viewed  under  a  narrow- 
band  light  source  even  though  the  MCC  patches  contain  many  different  hues. 

Consumer  cameras  carry  out  color  transformations  from  camera-dependent  color 
spaces  (i.e.  raw  image)  to  camera-independent  color  spaces  assuming  that  a  scene 
consists  of  reflectances  similar  to  those  in  a  standard  color  target,  and  that  the  ambient 
light  is  broadband  (e.g.,  daylight  or  one  of  common  indoor  illuminants),  because  most 
scenes  photographed  by  consumer  cameras  have  these  properties.  We  call  scenes  that  can 
be  represented  by  the  patches  of  a  standard  calibration  target  ordinary.  Non-ordinary 
scenes,  on  the  other  hand,  have  features  whose  reflectances  are  not  spanned  by 
calibration  target  patches  (e.g.  in  a  forest  there  may  be  many  shades  of  greens  and  browns 
that  common  calibration  targets  do  not  represent),  or  are  viewed  under  unusual  lighting 
(e.g.  under  monochromatic  light).  In  the  context  of  scientific  imaging  non-ordinary 
scenes  may  be  encountered  often;  we  give  examples  in  Ch  2.9. 

For  accurate  capture  of  colors  in  a  non-ordinary  scene,  a  color  calibration  target 
specific  to  that  scene  is  built.  This  is  not  a  physical  target  that  is  placed  in  the  scene  as 
described  in  Ch.  2.7;  instead,  it  is  a  matrix  containing  tri-stimulus  values  of  features  from 
that  scene.  Tri-stimulus  values  are  obtained  from  the  radiance  spectra  measured  from 
features  in  the  scene.  In  Figure  2.10  we  show  features  from  three  different  underwater 
habitats  from  which  spectra,  and  in  turn  tri-stimulus  values,  can  be  obtained. 

Spectra  are  converted  into  tri-stimulus  values  as  follows  (Reinhard  et  al.  2008): 

Xj  =  (2.10) 
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Figure  2.10  Features  from  three  different  dive  sites  that  could  he  used  for 
scene-specific  color  calihration.  This  image  first  appeared  in  the  December 
2012  issue  of  Sea  Technology  magazine. 
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where  =  X,X2  =  Y,  X^  =  Z,  and  K  =  Yi  yEi .  Here,  i  is  the  index  of  the  wavelength 
steps  at  which  data  were  recorded,  Ri  is  the  reflectance  spectrum  and  Ei  the  spectrum  of 
irradiance;  xj^  are  the  values  of  the  CIE  1931  color  matching  functions  x,  y,  z  at  the  ith 
wavelength  step,  respectively. 
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Figure  2.11  Scene-specific  color  transformation  improves  accuracy,  (a)  A  “non-ordinary”  scene 
that  has  no  chromaticity  overlap  with  the  patches  in  the  calibration  target,  (h)  Mean  error  after 
scene-specific  color  calibration  (SSCC)  is  significantly  less  than  after  using  a  calibration  chart, 
(c)  An  “ordinary”  scene  in  which  MCC  patches  span  the  chromaticities  in  the  scene,  (d) 
Resulting  error  between  the  MCC  and  scene-specific  color  transformation  is  comparable,  but  on 
average,  still  less  for  SSCC. 


Following  the  calculation  of  the  XYZ  tri-stimulus  values,  Eq.  2. 6-2. 8  can  be  used 
as  described  in  Ch.  2.7  to  perform  color  transformation.  However,  for  every  feature  in  a 
scene  whose  XYZ  values  are  calculated,  a  corresponding  RGB  triplet  that  represents  the 
camera  color  space  is  needed.  These  can  be  obtained  in  two  ways:  by  photographing  the 
features  at  the  time  of  spectral  data  collection,  or  by  simulating  the  RGB  values  using  the 
spectral  sensitivity  curves  of  the  camera  (if  they  are  known)  and  ambient  light  profile.  Eq. 
2.10  can  be  used  to  obtain  the  camera  RGB  values  by  substituting  the  camera  spectral 
sensitivity  curves  instead  of  the  color  matching  functions.  In  some  cases,  this  approach  is 
more  practical  than  taking  photographs  of  the  scene  features  (e.g.,  under  field  conditions 
when  light  may  be  varying  rapidly),  however  spectral  sensitivity  of  camera  sensors  is 
proprietary  and  not  made  available  by  most  manufacturers.  Manual  measurements  can  be 
done  through  the  use  of  a  monochromator  (Nakamura  2005),  a  set  of  narrowband 
interference  filters  (Mauer  and  Wueller  2009),  or  empirically  (Finlayson  et  al.  1998; 

Hong  et  al.  2001;  Barnard  and  Funt  2002;  Cheung  et  al.  2005;  Jiang  et  al.  2013). 
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2.8.1  Scene-specific  color  calibration  for  non-ordinary  scenes 

To  create  a  non-ordinary  seene,  we  used  292  natural  refleetanee  speetra  randomly 
seleeted  from  a  floral  refleetanee  database  (Arnold  et  al.  2010),  and  simulated  their 
radianee  with  the  underwater  light  profile  at  noon  shown  in  Figure  2.1  la  (seene  1).  While 
this  seems  like  an  unlikely  eombination,  it  allows  for  the  simulation  of  ehromatieity 
eoordinates  (Figure  2.1  la,  blaek  dots)  that  are  vastly  different  than  those  eorresponding 
to  an  MCC  under  noon  daylight  (Figure  2.1  la,  blaek  squares),  using  naturally  oeeurring 
light  and  refleetanees.  We  randomly  ehose  50%  of  the  floral  samples  to  be  in  the  training 
set  for  SSCC,  and  used  the  other  50%  as  a  novel  seene  for  testing.  When  this  novel  seene 
is  transformed  using  an  MCC,  the  mean  error  aeeording  to  Eq.  2.9  was  23.8  and  with 
SSCC,  it  was  1.56  (just  notieeable  differenee  threshold  is  1).  We  repeated  this 
transformation  100  times  to  ensure  test  and  training  sets  were  balaneed  and  found  that  the 
mean  error  values  remained  similar.  Note  that  the  resulting  low  error  with  SSCC  is  not 
due  to  the  high  number  of  floral  samples  used  (146)  in  training  for  SSCC,  eompared  to 
only  24  patehes  in  a  MCC.  Repeating  this  analysis  with  a  training  set  of  only  24 
randomly  seleeted  floral  samples  did  not  ehange  results  signifieantly. 

2.8.2  Scene-specific  color  calibration  for  ordinary  scenes 

We  used  the  same  speetra  from  seene  1  to  build  an  ordinary  seene  (seene  2),  i.e.  a 
seene  in  whieh  the  radianee  of  the  floral  samples  (Figure  2.1  le,  blaek  dots)  are  spanned 
by  the  radianee  of  the  patehes  of  an  MCC  (Figure  2.1  le,  blaek  squares).  In  this  ease,  the 
average  eolor  transformation  error  using  an  MCC  was  redueed  to  2.3;  but  it  was  higher 
than  the  error  obtained  using  SSCC  (Figure  2.  lid),  whieh  was  1 .73  when  24  patehes 
were  used  for  training,  and  1.5  with  146  patehes. 

Table  2  Summary  of  post-processing  steps  for  raw  images 


# 

Camera,  Light 

Demosaic 

White  balance 

Color  transformation 

I 

Sony  A700,  Incandescent  indoor 
light 

Adobe  DNG 
converter 

Version  6.3.0.79 
(for  list  of  other 
raw  image 
decoders,  see 
http://www.cyber 
com.net/~dcoffin/ 
dcraw/) 

4th  gray  &  black  in 
MCC 

Eq.  5 

None  -  analysis  in  the 
camera  color  space 

II 

Canon  EOS  IDs  Mark  II, 

Daylight 

MCC  and  SSCC 

III 

Canon  Rebel  T2,  Low-pressure 
sodium  light 

White  point  of 
ambient  light 
spectrum 

Eq.  5&10 

None  -  analysis  in  the 
camera  color  space 

IV 

Canon  EOS  IDs  Mark  II, 

Daylight 

SSCC 

V 

Canon  EOS  5D  Mark  II,  Daylight 
+  2  DS160  Ikelite  strobes 

4th  gray&  black  in 
MCC 

Eq.  5 

MCC 
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2.9  Examples  of  the  use  of  COTS  cameras  for  scientific  imaging 

Imaging  and  workflow  details  for  the  examples  in  this  seetion  are  given  in  Table  2. 

Example  I  -  Using  colors  from  a  photograph  to  quantify  temperature  distribution  on  a 
surface  (Figure  2.3  Steps:  1-3) 

With  careful 
calibration,  it  is 
possible  to  use 
inexpensive  cameras 
to  extract  reliable 
temperature  readings 
from  surfaces  painted 
with  temperature- 
sensitive  dyes,  whose 
emission  spectrum 
(color)  changes  when 
heated  or  cooled. 

Giirkan  et  al.  (Gurkan  et  al.  2012)  stained  the  channels  in  a  microchip  with  thermo¬ 
sensitive  dye  (Edmund  Scientific,  Tonawanda,  NY;  range  32°C  -  41°C)  and  locally 
heated  it  one  degree  at  a  time  using  thermo-electric  modules.  At  each  temperature  step  a 
set  of  baseline  raw  (linear  RGB)  photographs  were  taken.  Then,  novel  photographs  of  the 
chip  (also  in  raw  format)  were  taken  for  various  heating  and  cooling  scenarios.  Each 
novel  photograph  was  white  balanced  using  the  calibration  target  in  the  scene  (Eigure 
2.12).  Since  all  images  being  compared  were  acquired  with  the  same  camera,  colors  were 
kept  in  the  camera  color  space  and  no  color  transformation  was  applied.  To  get  the 
temperature  readings,  colors  along  the  microchip  channel  were  compared  to  the  baseline 
RGB  values  using  the  AE2000  metric  (Euo  et  al.  2001)  and  were  assigned  the  temperature 
of  the  nearest  baseline  color. 

Example  II:  Use  of  inexpensive  COTS  cameras  for  accurate  artwork  photography 
(Figure  2.3  Steps  1-4) 

Here,  we  quantify  the  error  associated  with  using  a  standard  calibration  target  (vs.  SSCC) 
for  imaging  an  oil  painting  (Eigure  2.13).  Low-cost,  easy-to-use  consumer  cameras  and 
standard  color  calibration  targets  are  often  used  in  artwork  archival;  a  complex  and 
specialized  application  to  which  many  fine  art  and  cultural  heritage  institutions  allocate 
considerable  resources.  Though  many  museums  in  the  Unites  States  have  been  using 
digital  photography  for  direct  capture  of  their  artwork  since  the  late  1990s  (Rosen  and 
Erey  2005),  Erey  and  colleagues  (Erey  and  Earnand  2011)  found  that  some  institutions 
did  not  include  color  calibration  targets  in  their  imaging  workflows  at  all.  Eor  this 
example,  radiance  from  36  points  across  the  painting  were  measured,  and  it  was  found 
that  their  corresponding  tri-stimulus  values  were  within  the  span  of  the  subspace  of  the 
MCC  patches  under  identical  light  conditions,  i.e.  an  ordinary  scene.  The  MCC-based 
color  transformation  yielded  an  average  AE2000  value  of  0.21,  and  0.22  for  SSCC,  both 
below  the  just  noticeable  difference  threshold  of  1.  However,  the  average  error  was  31.77 
for  the  jpg  output  produced  by  the  camera,  used  in  auto  setting.  Eor  this  ordinary  scene, 
there  was  no  advantage  to  be  gained  from  SSCC  and  the  use  of  a  standard  calibration 


Figure  2.12  Temperature  distribution  along  the  microchip  channel  which  is 
locally  heated  to  39°C  (colored  region)  while  the  rest  was  kept  below  32°C 
(black  region). 
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target  with  the  workflow  in  Figure  2.3  significantly  improved  color  accuracy  over  the  in¬ 
camera  processed  image. 
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Figure  2.13  Example  II:  Use  of  inexpensive  COTS  cameras  for  accurate  artwork  photography, 
(a)  An  oil  painting  under  daylight  illumination,  (b)  Thirty-six  points  from  which  ground  truth 
spectra  were  measured,  (b)  Chromatic  loci  of  the  ground  truth  samples  compared  to  Macbeth 
ColorChecker  patches,  under  identical  illumination,  (d)  sRGB  representation  of  the  colors  used 
for  scene-specific  calibration.  Artwork:  Fulya  Akkaynak. 


Example  III  -  Capturing  photographs  under  monochromatic  low  pressure  sodium  light 
(Figure  2.3  Steps  1-3) 

A  monochromatic  light  spectrum  E  can  be  approximated  by  an  impulse  centered  at  the 
peak  wavelength  Ap  as  E  —  C  ■  5  (A  —  Ap)  where  C  is  the  magnitude  of  intensity,  and  5 
is  the  Dirac  delta  function  whose  value  is  zero  everywhere  except  when  X  =  Ap.  When 
A  =  Ip,  5  =  1.  It  is  not  trivial  to  capture  monochromatic  scenes  using  COTS  cameras 
accurately,  because  they  are  optimized  for  use  with  indoor  and  outdoor  broadband  light 
sources.  We  used  low-pressure  sodium  light  (Ip  =  589  nm)  to  investigate  the  effect  of 
color  (or  lack  of)  on  visual  perception  of  the  material  properties  of  objects.  Any  surface 
viewed  under  this  light  source  appears  as  a  shade  of  orange  because  =  z  =  0  at 

1  =  589  nm  for  the  CIE  XYZ  human  color  matching  functions  (also  for  most  consumer 
cameras),  and  in  turn,  —  Z  =  0  at  1  =  589  nm  (Eq.  2.10).  Subjects  were  asked  to 
view  real  textiles  once  under  sodium  light  and  once  under  broadband  light  and  answer 
questions  about  their  material  properties.  The  experiment  was  then  repeated  using 
photographs  of  the  same  textiles.  To  capture  the  appearance  of  colors  under  sodium  light 
accurately,  its  irradiance  was  recorded  using  a  spectrometer  fitted  with  an  irradiance 
probe.  Then,  the  tri-stimulus  values  corresponding  to  its  white  point  were  calculated  and 
used  for  RGB-equalization  of  linear  novel  images.  Due  to  the  monochromatic  nature  of 
the  light  source,  there  was  no  need  to  also  apply  a  color  transformation;  adjusting  the 
white  point  of  the  illuminant  in  the  images  ensured  that  the  single  hue  in  the  scene  was 
mapped  correctly.  The  white  point  of  the  illuminant  was  indeed  captured  incorrectly 
when  the  camera  was  operated  in  auto  mode,  and  the  appearance  of  the  textiles  was 
noticeably  different  (Eigure  2.14). 
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Figure  2.14  Example  III  Capturing  photographs  under  monochromatic  low-pressure 
sodium  light,  (a)  A  pair  of  fabrics  under  hroadhand  light,  (h)  A  jpg  image  taken  with  the 
auto  settings  of  a  camera,  under  monochromatic  sodium  light,  (c)  Image  processed  using 
scene-specific  color  calihration  according  to  the  flow  in  Figure  2.3. 


Example  IV:  In  situ  capture  of  a  camouflaged  animal  and  its  habitat  underwater  (Figure 
2.3  Steps  1-4) 

Imaging  underwater  is  challenging  because  light  is  fundamentally  different  from  what  we 
encounter  on  land  (Figure 
2.5)  and  each  underwater 
habitat  is  different  from 
another  in  terms  of  the 
colorfulness  of  substrates 
(Figure  2.10).  Thus,  there 
is  no  global  color  chart 
that  can  be  used  across 
underwater  environments, 
making  underwater 
scenes  (except  for  those 
very  close  to  the  surface) 
non-ordinary  scenes.  The 
majority  of  underwater 
imaging  falls  into  three 
categories  motivated  by 
different  requirements: 

(1)  using  natural  light  to 
capture  a  scene  exactly 
the  way  it  appears  at  a 
given  depth  (2)  using 

natural  light  to  capture  an  underwater  scene  but  post-processing  to  obtain  its  appearance 
on  land  and  (3)  introducing  artificial  broadband  light,  (e.g.,  (Vasilescu  2009),  in  situ  to 
capture  the  scene  as  it  would  have  appeared  on  land  (Example  V).  Here  we  give  an 
example  for  case  (1):  capture  of  the  colors  of  the  skin  of  a  camouflaged  animal  exactly 
the  way  an  observer  in  situ  would  have  seen  them.  For  this  application,  we  surveyed  a 
dive  site  and  built  a  database  of  substrate  reflectance  and  light  spectra  (Akkaynak  et  al. 
2013).  Novel  (raw)  images  taken  at  the  same  site  were  white  balanced  to  match  the  white 
point  of  the  ambient  light.  We  then  used  the  spectral  database  built  for  this  dive  site  for 
SSCC  (Figure  2.15).  The  green  hue  in  the  resulting  images  may  appear  unusual  for  two 


(0  (d) 

Figure  2.15  Example  IV  In  situ  capture  of  (a)  an  underwater  habitat  and 
(b)  a  camouflaged  cuttlefish  using  scene-specific  color  calibration  with 
features  similar  to  those  shown  in  Fig.  10  for  Urla,  Turkey,  (c)  and  (d) 
are  jpg  outputs  directly  from  the  camera  operated  in  auto  mode  and  have 
a  visible  red  tint  as  a  consequence  of  in-camera  processing. 
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reasons.  First,  a  light-adapted  human  diver  may  not  have  perceived  that  green  color  in 
situ  due  to  color  constancy  and  therefore  may  not  remember  the  scene  to  appear  that  way. 
Second,  most  professional  underwater  photographs  we  are  familiar  with  are  either  post- 
processed  to  appear  less  green  (case  2),  or  they  are  taken  with  strobes  that  introduce 
broadband  light  to  the  scene  to  cancel  the  green  appearance  before  image  capture  (case 
3).  This  kind  of  processing  is  indeed  similar  to  the  processing  that  may  be  performed  by  a 
COTS  camera;  the  camera  will  assume  a  common  land  light  prohle  (unless  it  is  being 
operated  in  a  pre-programmed  underwater  mode),  ignorant  of  the  long-wavelength 
attenuated  ambient  light,  and  this  will  result  in  boosting  of  the  red  channel  pixel  values. 

Example  V:  Consistent  underwater  color  correction  across  multiple  images  (Figure  2.3 
Steps:  1-4) 

Repeated  consistent 
measurements  are  the 
foundation  of  ecological 
monitoring  projects.  In 
the  case  of  corals,  color 
information  can  help 
distinguish  between 
different  algal  functional 
groups,  or  even  establish 
coral  health  change 
(Siebeck  et  al.  2006). 

Until  now,  several  color 
correction  targets  were 
used  for  coral  monitoring 
(Winters  et  al.  2009),  but 
not  tested  for  consistency.  To  test  our  method’s  performance  for  consistent  color  capture, 
we  attached  one  color  chart  to  the  camera  using  a  monopod,  and  placed  another  one 
manually  in  different  locations  in  the  field  of  view  (Fig.  17).  We  tested  several  correction 
algorithms:  automatic  adjustment,  white  balancing  and  the  algorithm  presented  in  this 
paper  (Fig.  3)  and  defined  the  error  as  consistency  of  the  corrected  color.  For  each  of  the 
M=35  patches  in  the  N=68  corrected  images,  we  calculated  mean  chromaticity  as: 

fin  ~  ^2n=l  hn,n  >  9m  ~  9m, n  >  ^  M  where  T  —  R/ (R  -f  (7  -f  5),  g  — 

G/(R  +  G  +  B)  and  defined  the  patch  error  to  be  (fin.t'  9m, i)  -  (%-  || 

with  total  error  as  the  average  across  ah  M  patches.  Our  algorithm  yielded  the  most 
consistent  colors,  opening  the  possibility  for  the  use  of  color  targets  designed  for  land 
photography  in  marine  ecology  images. 
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Figure  2.16  Example  V  Consistent  underwater  color  correction,  (a)  Top 
row:  six  of  the  68  raw  images  taken  in  clear  water  in  Totoya  reef,  Fiji, 
before  color  correction;  bottom  row:  after.  In  each  frame,  the  color 
chart  on  the  left  was  used  for  calibration,  and  the  one  on  the  right  for 
testing,  (b)  Average  error  for  several  color  correction  method  for  the 
calibration  (blue)  and  the  test  (red)  chart.  Our  method  achieves  the 
lowest  error  and  is  the  only  method  to  improve  over  the  raw  images  of 
the  test  chart. 
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2.10  Conclusion 

Concepts  discussed  in  this  work  are  central  topics  in  research  fields  such  as 
applied  optics,  computational  photography,  optical  imaging  and  colorimetry.  While 
professional  photographers  and  researchers  in  fields  like  computer  vision  frequently  use 
raw  images  and  calibration  targets  to  transform  device-dependent  colors  to  a  device¬ 
independent  space,  the  importance  of  manually  calibrating  consumer  camera  images  has 
only  recently  become  understood  by  animal  biology  scholars.  We  streamlined  these 
calibration  steps  in  a  simple  and  direct  manner  to  make  them  accessible  to  researchers 
from  all  areas,  without  assuming  familiarity  with  imaging-related  sciences  or  the 
technology  behind  consumer  cameras. 

Four  steps  are  necessary  for  the  use  of  consumer  cameras  as  consistent  and 
accurate  data  acquisition  instruments:  raw  image  capture,  demosaicing  of  the  raw  image, 
white  balancing  and  color  transformation  (Figure  2.3).  How  each  step  of  processing 
should  be  executed  depends  on  the  particular  application;  we  have  given  examples  in 
Chapter  2.9.  When  pixel  intensity  values  are  to  be  used  as  scientific  data,  at  the  very  least 
an  off-the-shelf  calibration  target  should  used  to  obtain  consistent  colors  to  allow  for 
reproducibility  of  work  by  other  researchers  with  no  knowledge  of  the  particular  imaging 
instrument.  If  accurate  color  capture  is  also  desired  in  addition  to  consistency,  reflectance 
and  irradiance  spectra  (the  product  of  which  yields  radiance)  collected  from  features  from 
the  scene  can  be  used  to  obtain  tristimulus  values,  forming  the  baseline  for  a  scene- 
specific  calibration  target.  A  scene-specific  approach  to  color  transformation  is  especially 
important  when  the  scene  contains  objects  whose  chromaticities  are  vastly  different  from 
those  in  an  average  scene  we  encounter  on  land,  or  if  it  was  viewed  under  a  narrow- 
broadband  light,  or  both.  In  the  case  of  a  simulated  underwater  scene,  we  showed  that 
color  transform  could  be  achieved  with  approximately  a  20-fold  reduction  in  error  with 
scene-specific  color  calibration. 

We  recognize  that  the  methodology  described  here  adds  some  complication  to  the 
very  feature  that  makes  COTS  cameras  popular  -  ease  of  use.  To  allow  for  easier 
adoption  of  the  extra  steps  required  and  their  seamless  integration  into  research  projects, 
we  provide  a  toolbox  of  functions  written  in  MATLAB  programming  language 
(Mathworks,  Inc.  Natick  MA)  available  for  download  at  the  following  link: 
http://www.mathworks.com/matlabcentral/fileexchange/42548  .  For  some  steps  in  our 
workflow,  knowing  the  spectral  sensitivities  of  a  specific  camera  is  advantageous. 

Camera  manufacturers  do  not  provide  these  curves  and  the  burden  falls  on  the  users  to 
derive  them.  Carrying  out  a  survey  to  collect  spectral  data  from  a  site  to  perform  scene- 
specific  color  calibration  requires  the  use  of  a  spectroscopic  imaging  device,  which  adds 
to  the  overall  cost  of  the  study  and  may  extend  its  duration.  Yet,  despite  the  extra  effort 
required  to  calibrate  them  for  scientific  purposes,  COTS  cameras  can  have  more  utility 
than  hyper-spectral  imagers  or  spectrometers  depending  for  applications  requiring  image 
capture  in  the  visible  range. 


37 


Chapter  3 

Unbiased  data  acquisition:  Spectrometers^ 

3.1  Introduction 

In  the  field  of  animal  eoloration  and  eamoufiage,  speetrometers  with  optieal  fibers 
are  frequently  used  to  measure  eolors  from  an  animal’s  habitat,  nest,  eggs,  body  parts, 
skin,  fur  and  plumage  (Table  3).  Speetra  measured  from  these  features  are  then  used  in 
models  that  quantify  eolor  diseriminability  aeeording  to  human  or  non-human  visual 
systems.  In  terms  of  eolor  eapture,  speetrometers  have  a  elear  advantage  over  eonsumer 
eameras  (see  Chapter  2)  beeause  they  densely  sample  the  eleetromagnetie  speetrum, 
typieally  imaging  between  70-100  bands,  but  often  a  thousand  or  more.  In  addition,  they 
image  outside  the  visible  range,  whieh  most  eonsumer  eameras  eannot.  However, 
speetrometers  eolleet  data  in  a  point-by-point  fashion  whieh  means  that  they  are  not 
suitable  for  reeording  eolors  from  textured  surfaees,  i.e.  those  with  high  frequeney  eolor 
or  pattern  elements.  Even  in  the  ease  of  surfaees  with  solid  colors  and  a  carefully  gridded, 
sequential  data  collection  setup,  the  distance  of  the  optical  fiber  to  the  sample,  its 
diameter,  numerical  aperture  and  the  measurement  angle  could  affect  the  purity  of  the 
color  signal  recorded.  If  a  spectral  measurement  is  contaminated  with  neighboring  colors, 
our  assessment  of  the  diseriminability  of  that  spectrum  from  another  will  no  doubt  be 
inaccurate.  Given  an  optical  fiber  of  known  diameter  and  numerical  aperture,  and  a  body 
pattern  feature  of  certain  size,  how  can  we  quantify  the  degree  of  spectral  contamination 
of  our  measurements? 

In  general,  it  is  known  to  researchers  that  measurements  should  be  taken  as  close 
to  the  surface  being  measured  as  possible.  For  laboratory  measurements,  optical  fibers  are 
fixed  inside  probe  holders  that  have  machined  45  and  90-degree  slots,  securing  the  fiber 
at  a  distance  that  is  nearly  touching  the  sample  at  the  desired  angle.  For  spectroscopic 
measurements  in  the  field,  this  is  very  difficult  to  establish;  recording  spectra  closely 
from  live  animals  is  very  challenging  (Figure  3.1).  In  addition,  ambient  light  conditions 
may  change  before  a  measurement  is  completed.  The  current  standard  for  researchers 
conducting  spectroscopic  studies  in  the  field  is  to  report  an  estimate  of  the  distance  and 
angle  of  the  optical  fiber  in  their  publications  (Table  3).  Inadvertently,  not  being  able  to 
get  close  enough  to  the  animals,  and  then  having  to  estimate  the  measurement  distance 
results  in  errors  regarding  diseriminability  of  colors  for  camouflage  assessment.  In  this 
chapter,  I  develop  an  equation  using  geometrical  optics  that  quantifies  what  these  errors 
may  be  in  the  context  of  animal  coloration.  This  equation  depends  only  on  the  diameter 
and  the  numerical  aperture  of  the  optical  fiber.  Using  the  visual  systems  of  a  dichromatic 
ferret  {Mustela  putorious  furo)  and  a  trichromatic  frog  {Dendrobates  pumilio),  I  test  this 
equation  on  a  pair  of  colors:  (1)  that  have  little  spectral  overlap  simulating  a  conspicuous 
appearance,  and  (2)  that  have  high  spectral  overlap  simulating  a  camouflage  pattern. 
Modeling  a  visual  system  that  has  one  more  photoreceptor  than  the  di-chromat  will  help 
investigate  how  potential  errors  contaminated  spectra  cause  translate  into  different 
assessments  of  color  diseriminability. 


2 

Parts  of  this  chapter  was  previously  published.  See  (Akkaynak  2014)  for  full  reference. 
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3.2  Background  and  related  work 

One  aspect  of  color  vision  is  the  ability  to  distinguish  surfaces  based  on  their 
spectral  properties  (Brainard  2001).  In  the  field  of  animal  biology,  understanding  animal 
color  vision  helps  answer  important  scientific  questions.  For  example,  Maan  and 
Cummings  (Maan  and  Cummings  2008)  found  that  for  female  poison  frogs,  the  male’s 
dorsal  color  is  the  only  factor  determining  their  choice  of  a  mate.  In  addition  to  sexual 
selection,  animals  use  color  vision  for  behavioral  tasks  including  finding  food, 
recognizing  objects  and  communicating  with  members  of  the  same  species  (Kelber  et  al. 
2003).  To  accurately  capture  and  interpret  how  color  signals  are  received,  we  must  have  a 
model  of  the  observer  animal’s  visual  system.  Analyzing  color  signals  from  the  point-of- 
view  of  humans  and  making  inferences  about  their  appearances  to  animals  often  produces 
erroneous  conclusions.  For  example  Stoddard  and  Stevens  (Stoddard  and  Stevens  2011) 
showed  that  some  common  cuckoo  eggs  (which  are  laid  in  nests  of  other  species  and 
often  hatched  and  raised  by  unsuspecting  host  birds)  appeared  to  have  a  good  color  match 
to  the  host  eggs  when  viewed  by  humans,  but  had  clear  and  quantifiable  differences  when 
modeled  in  avian  color  space.  A  detailed  summary  of  how  color  signals  can  be  modeled 
from  the  perspective  of  animals  can  be  found  in  (Endler  and  Mielke  Jr  2005).  In  this 
work,  my  focus  is  not  on  these  models,  but  on  the  measurement  of  the  color  signals. 

Color  is  inherently  difficult  to  measure  objectively,  because  it  is  a  sensation  defined 
relative  to  the  human  visual  system  (Wyszecki  and  Stiles  2000).  Visual  systems  of  most 
other  animals  are  different  from  our,  and  analyses  from  the  perspective  of  other  systems 
must  start  with  unfiltered  spectral  data,  encompassing  the  segment  of  the  electromagnetic 
spectrum  to  which  the  animal  is  sensitive  (see  Chapter  2).  Spectral  information  in  a  scene 
can  most  comprehensively  be  captured  using  a  hyperspectral  imager  (Akkaynak  et  al. 
2014),  which,  for  each  pixel  in  its  sensor,  records  a  densely  sampled  portion  of  the 
electromagnetic  spectrum.  Despite  providing  spectral  data  for  every  pixel  in  an  image, 
hyperspectral  imagers  are  not  commonly  used  in  studies  of  animal  coloration  (or  most 
other  research  fields).  This  is  because  they  are  still  expensive,  physically  bulky,  have  low 
resolution,  create  large  digital  files  and  are  not  fast  enough  for  imaging  moving  objects 
(i.e.  animals  in  the  wild).  Although  consumer  digital  cameras  are  inexpensive,  fast, 
compact  and  widely  available,  they  cannot  serve  as  substitutes  for  hyperspectral  imagers 
because  they  record  spectral  information  with  limited  spectral  resolution  (Akkaynak  et  al. 
2014).  Thus,  spectrometers,  which  are  effectively  single-pixel  hyperspectral  imagers, 
provide  a  compact  and  affordable  solution  for  animal  coloration  studies;  some  examples 
of  recent  studies  measuring  animal  or  habitat  spectra  and  then  modeling  discriminability 
in  the  eyes  of  animal  visual  systems  are  given  in  Table  3. 

Most  spectroscopic  studies  are  conducted  in  laboratory  settings  using  carefully 
designed  optical  benches.  Key  elements  of  these  setups  are  fiber  holders,  which  are 
essentially  guides  that  ensure  spectra  from  the  sample  are  measured  as  close  to  its  surface 
as  possible  without  making  contact.  For  free-hand  spectroscopic  measurements,  how 
close  the  optical  fiber  should  be  held  at,  and  at  what  angle,  for  a  feature  of  known  size 
has  not  previously  been  quantified;  that  is  described  next. 
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Table  3  Recent  publications  that  assess  the  discriminability  of  colors  relative  based  on 
measurements  taken  by  spectrometers  (partial  list). 


Publication 

Fiber  Distance  & 

Angle 

Spectra  Measured 

Visual 

System 

Modeled 

(Chiao  et  al.  2011) 

Hyperspectral 

N/A 

European  cuttlefish 

Fish 

(Isaac  and  Gregory  2013) 

Spectrometer 

100  pm 

2  mm/ 90° 

Western  terrestrial 
garter  snake 

Ferret, 
human,  birds 

(Akkaynak  et  al.  20 1 3) 

Spectrometer 

50  and  100  pm 
<  3  cm 

European  cuttlefish 

Fish,  human 

(Hanlon  et  al.  2013) 

Spectrometer 

400  pm 

2  cm 

Giant  Australian 
cuttlefish 

Fish 

(Finkbeiner  et  al.  2012) 

Spectrometer 

400  pm 

45° 

Passion-vine 

butterflies 

Birds 

(Lind  et  al.  2013) 

Spectrometer 

1000  pm 

45° 

Blue  tit 

Birds 

(Maan  and  Cummings  2012) 

Spectrometer 

400  pm 

3mm/90° 

Strawberry,  poison 
frog 

Birds,  crabs, 
snakes,  frogs 

(Bybee  et  al.  2012) 

Spectrometer 

400  pm 

45° 

Heliconius  butterfly 

Birds, 

butterflies 

(Cortesi  and  Cheney  20 1 0) 

Spectrometer 

200  pm 

45° 

Marine  opisthobranchs 

Fish 

(Langmore  et  al.  2009) 

Spectrometer 

N/A 

45° 

Cuckoo  eggshell 

Birds 

(Baldwin  and  Johnsen  2012) 

Spectrometer 

400  pm 

45° 

Blue  crab 

Blue  crab 

(Nokelainen  et  al.  2012) 

Spectrometer 

N/A 

45-90° 

Wood  tiger  moth 

Blue  tit 

(Stoddard  et  al.  2011) 

Spectrometer 

N/A 

45° 

Cuckoo  eggs 

Birds 

(Siddiqi  et  al.  2004) 

Spectrometer 

N/A 

N/A 

Strawberry  poison 
frogs 

Strawberry 
poison  frogs 
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3.3  Finding  optimal  measurement  distance  and  angle 

An  optical  fiber  is  a  waveguide  that  propagates  rays  through  total  internal 
reflection  (Bass  et  al.  2009).  It  is  a  cylindrically  symmetric  cable  with  a  core  that  has  high 
refractive  index,  Ucore^  surrounded  by  eladding  with  low  refractive  index,  riciad  (Figure 
3.2).  Fibers  are  characterized  by  their  numerical  aperture  (NA),  which  is  defined  relative 
to  the  refractive  index  of  the  medium  of  operation  {rimed)  the  aeceptance  angle  (0o) 


(a) 


(b) 


Figure  3.1  (a)  A  SCUBA  diver  extending  the  optical  fiber  attached  to  a  spectrometer  (in 
water  and  pressure  proof  housing)  to  record  the  spectrum  of  light  reflected  from  the  skin 
of  a  cuttlefish,  in  Urla,  Turkey.  Getting  the  optical  fiber  close  enough  to  take  accurate 
measurements  from  freely  behaving  animals  in  the  wild  is  challenging.  Photo  credit: 

Derya  Akkaynak.  (b)  Spectral  measurements  of  a  specimen  can  be  recorded  in  a 
laboratory  by  getting  the  fiber  as  close  to  the  specimen  as  possible  without  touching  it. 
Photo  courtesy  of  M.C.  Stoddard  &  K.  Zyskowski,  taken  at  the  Ornithology  Collections  of 
the  Peabody  Museum  of  Natural  History,  Yale  University,  New  Haven  CT. 


as: 


NA  =  rimed  ■  sin  Oq 


(3.1) 


Acceptance  angle  is  the  largest  incidence  angle  for  incoming  rays  to  undergo  total 
internal  refleetion.  The  locus  of  all  such  rays  is  the  cone  of  acceptance  (shown  in  light 
gray  in  Figure  3.2).  For  most  animal  biology  applications,  the  optical  fiber  is  used  in  air 
{rimed  ~  or  water  {rimed  ~  1-33);  the  difference  between  fresh  and  salt  water  is 
usually  ignored  for  praetieal  purposes. 

When  the  fiber  is  oriented  along  the  Z-direction  and  is  held  at  distance  of  d  from  a 
surface  tilted  at  an  angle  a  relative  to  the  XY  plane,  the  eross-section  of  the  aeeeptanee 
cone  on  the  surface  is  an  ellipse  (Figure  3.2).  In  the  trivial  case  when  a  =  0,  the  plane 
will  be  parallel  to  the  base  of  the  eone,  and  interseetion  of  the  eone  and  the  plane  will  be 
a  circle.  In  the  general  ease  (derivation  is  given  in  Appendix),  for  a  fiber  with  diameter  df 
and  NA,  the  elliptical  cross  section  is  in  the  form 


(3.2) 


where 


(3.3) 
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are  the  eoordinates  of  the  eenter  of  the  ellipse,  and 


a 


2 


and 


b^=- 


(3.4) 


(3.5) 


are  the  squares  of  the  semi -major  and  semi-minor  axes,  respeetively.  Here,  it  ean  be 
shown  from  geometry  that  t  =  -  =  tan(sln“^(iVi4/n^gd)).  The  height  is  measured 
from  the  apex  of  the  eone  to  the  interseetion  of  the  surface  and  the  axis  of  the  optical 
fiber  (Figure  3.2)  and  is  given  by: 


Hs  =  d  + 


_ If _ 

2  tan  (sin-1  (Wi4/Timed)) 


(3.6) 


surface  being  measured 


nmcd 


^4: 

^  X 


■Ocla'dl 


3.4  Application  to  animal  color  discrimination 

In  this  section,  the 
effect  of  measurement  distance 
and  angle  on  the  quality  of  the 
recorded  spectral  signals  is 
investigated  using  the  visual 
systems  of  a  dichromatic  ferret 
(Mustela  putorious  furo)  and  a 
trichromatic  frog  (Dendrobates 
pumilio).  Fibers  are  assumed  to 
have  diameters  100, 400  and 
1000  jum  based  on  those 
commonly  used  (Table  3),  with 
NA  —  0.22.  The  operation 
medium  is  air  (Urned  —  1) 
the  illuminant  is  CIE  D65, 
which  is  a  theoretical  light 

spectrum  that  approximates  noon  daylight.  Solid  color  patches  of  dimensions  4.1  cm  x 
4.1  cm  (the  actual  size  of  patches  on  a  Macbeth  ColorChecker,  Xrite,  Inc.)  on  a  synthetic 
test  target  are  used  to  simulate  two  test  cases  relevant  to  animal  coloration:  (1)  colors 
expected  to  yield  high  contrast  (i.e.  a  conspicuous  animal)  and  (2)  colors  expected  to 
yield  low  contrast  (i.e.  a  camouflaged  animal). 


[  d.  \ 

Hcorc 

r 

cone  of 
acceptance 

1 

\ 

1 

Go:  acceptance  angle 
n:  refractive  index 

d:  measurement 
distance 

dr  fiber  diameter 

Hsi  cone  height  at  the 

'  d  ' 

Hs 

H 

intersection  of  the 
surface 

H:  cone  height 
R:  base  radius  at  H 

Figure  3.2  Field  of  view  of  an  optical  fiber. 
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3.4.1  Synthetic  test  stimuli 

A  synthetic  multi-spectral  test  target  was  created  to  show  the  effect  of  sampling 
colors  erroneously.  The  target  was  designed  to  have  the  same  arrangement  and  patch  size 
as  a  Macbeth  ColorChecker,  a  standard  in  color  calibration.  Since  Macbeth  charts  only 
provide  tristimulus  values,  complete  spectra  from  the  METACOW  project  (Fairchild  and 
Johnson  2004)  were  substituted  for  matching  colors.  The  spectra  were  obtained  by 
averaging  the  reflectance  spectra  of  a  50  x  50  pixel  region  from  the  left  half  of  each 
METACOW  sample.  The  resulting  stimulus  is  a  hyperspectral  calibration  target  with 
square  patches  of  solid  colors,  each  of  which  has  reflectance  spectra  in  the  interval  400- 
700  nm  in  5  nm  steps  (Figure  3.3). 

For  case  (1),  spectra  from  patches  A  and  B  (Figure  3.3)  were  chosen  to  represent 
a  conspicuous  animal’s  pattern.  The  red  and  blue  patches  are  expected  to  contrast  highly 
for  any  visual  system  because  they  have  little  spectral  overlap.  To  create  a  camouflaged 
animal  that  perfectly  matches  the  colors  of  its  background  for  case  (2),  the  reflectance 
spectrum  of  patch  A  was  copied  onto  the  location  of  patch  B,  creating  two  neighboring 
patches  that  are  identical.  These  patches  would  be  expected  to  have  minimal  color 
contrast  if  the  color  was  sampled  without  contamination  from  neighboring  colors.  Note 
that  in  both  cases,  there  is  black  border  between  color  patches  (similar  to  the  original 
Macbeth  ColorChecker),  which  can  affect  the  quality  of  the  recorded  signal. 


3.4.2  Mathematical  similarity  of  two  spectra 

It  is  often  useful  to  assess  the  similarity  (or,  dissimilarity)  of  two  spectra  without 
referencing  a  biological  visual  system  (Akkaynak  et  al.  2013).  In  this  paper,  the 
mathematical  dissimilarity  between  the  pure  and  recorded  version  of  a  spectrum  is  used 
as  an  objective  measure  of  contamination,  which  is  quantified  using  the  Spectral  Angle 
Mapper  (SAM)  metric  (Yuhas  et  al.  1992): 


^SAM 


cos 


SaSb 


(3.7) 


Here,  spectra  are  treated  as  two  high  dimensional  vectors  (S^,  S^).  The  angle  between  the 
vectors  can  be  thought  of  a  measure  of  alignment;  the  smaller  the  angle,  the  more  similar 
the  two  spectra  are  in  shape.  This  metric  is  insensitive  to  differences  in  magnitude. 

3.4.3  Modeling  of  color  discriminability 


wavelength{nm) 

(b) 


Figure  3.3  Test  stimuli  used  for  assessment  of  color  discrimination  for  a  conspicuous  animal 
(case  f),  and  a  camouflaged  animal  (case  2).  (a).  The  synthetic  hyperspectral  test  target  has  the 
same  layout  as  the  Macheth  ColorChecker  shown  here.  Patches  A  and  B  are  those  selected  to  he 
highly  contrasting  with  each  other  for  case  (f).  For  case  (2),  the  reflectance  of  patch  A  is  copied 
to  the  location  of  patch  B,  creating  two  patches  with  identical  spectra  next  to  each  other, 
separated  hy  a  hlack  border,  (h)  Reflectance  spectra  of  patches  A  and  B. 
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One  of  the  methods  frequently  used  to  assess  diseriminability  of  colors  in  animal 
vision  is  the  receptor  noise  model  developed  by  Vorobyev  and  Osorio  (Vorobyev  and 
Osorio  1998).  In  the  absence  of  perceptual  thresholds  for  the  visual  systems  of  most 
animals,  this  model  predicts  color  diseriminability  by  assuming  that  thresholds  are  set  by 
color  opponency  mechanisms  whose  performance  is  limited  by  receptor  noise.  For  a 
visual  system  with  n  receptor  channels,  color  is  encoded  with  n-1  opponent  mechanisms; 
the  achromatic  signal  is  disregarded  and  the  relative  proportion  of  each  receptor 
determines  the  receptor  noise.  Similar  to  the  CIE  LAB-based  distance  metrics,  two  color 
stimuli  are  indistinguishable  if  the  distance  between  them  is  one  just  noticeable  difference 
(JND)  or  less.  According  to  this  model,  color  contrast  (A5)  for  a  dichromat  visual  system 
is  computed  as  follows  (Chiao  et  al.  2011): 


2  ^ 

ei+ef 

and  for  a  trichromat: 


(3.8) 


(eie2)2+(eie3)2+(e2e3)2  ’  '' 

where  AQj  =  Qi  a  —  Qi,b  is  the  difference  between  the  quantum  catch  Qi  of  a  stimuli  a 
and  b  in  the  photoreceptor  type.  Quantum  catch  is  found  by 

G I  owmwdA,  (3.10) 

where  and  Imax  are  the  bounds  of  the  light  spectrum  of  interest,  7(1)  is  the 
spectrum  of  the  incident  light,  77(1)  is  the  reflectance  spectrum  of  the  surface  being 
measured,  5j(l)  is  the  spectral  sensitivity  of  receptor  type  i,  and  k  is  the  von  Kries 
adaptation  constant,  which  is  set  to  unity  for  this  paper.  The  gj  in  Eq.  3.8&3.9  represents 
noise  in  receptor  type  i  and  can  be  approximated  using  the  Weber  fraction  (wQ  (Chiao  et 
al.  2011): 


where  rij  is  the  estimate  of  the  proportion  of  the  7*  wavelength  sensitive  cone  (i.e.  short, 
medium,  or  long)  and  is  the  estimate  of  the  proportion  of  the  long  wavelength 
sensitive  cones  (Chiao  et  al.  2011). 

Eor  the  visual  system  parameters  for  the  poison  dart  frog  are  taken  from  (Siddiqi 
et  al.  2004)  (k^ax,s,M,L  —  470, 489,  561  nm;  —  ^>^sws  —  1)  and  those 

for  the  ferret  are  from  (Isaac  and  Gregory  20I3)(l^a;f  5^  =  430,  558  nm;  — 

14,  —  l).  The  spectral  sensitivities  corresponding  to  the  Imax  values  were 

calculated  as  described  in  (Akkaynak  et  al.  2013). 
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Figure  3.4  (a)  When  the  optical  flher  (NA  =  0.22)  is  held  at  90°  to  the  surface  being  measured, 
the  cross  section  of  the  cone  of  acceptance  is  a  disk.  As  the  measurement  distance  increases, 
the  radius  of  the  disk  also  increases.  For  a  measurement  at  45°,  the  cross  section  is  an  ellipse. 
The  concentric  curves  show  cross-sections  at  distances  0-20  cm,  in  10  equal  steps,  (h) 
Mathematical  similarity  of  patches  A  (hlack  lines)  and  B  (gray  lines)  to  their  uncontaminated 
versions.  Note  that  SAM  only  measures  the  similarity  of  the  shape  of  the  two  spectra, 
disregarding  magnitude.  Results  for  different  fiber  diameters  were  similar  and  so  only  those 
for  100  pm  are  shown  here. 


3.5  Results 

In  this  section,  the  effect  of  d  and  a  on  the  spectral  content  of  a  signal  is 
investigated  by  comparing  the  signal  to  its  uncontaminated  reflectance  spectrum 
mathematically  and  then  perceptually  relative  to  two  visual  systems.  For  simplicity,  it  is 
assumed  that  the  spectral  measurements  for  the  test  stimuli  are  taken  with  the  tip  of  the 
fiber  pointing  to  the  center  of  each  patch,  and  that  when  measurements  for  two  patches 
are  being  compared,  they  were  recorded  at  the  same  d  and  a. 

3. 5. 1  Mathematical  similarity  of  spectra 

In  Figure  3.4a  cross-sections  of  the  cone  of  acceptance  are  shown  for  two 
measurement  angles  that  are  commonly  used  in  animal  coloration  measurements,  90° 
(cross-sections  are  circles)  and  45°  (cross-sections  are  ellipses),  respectively,  with 
measurement  distance  varying  from  0  to  20  cm.  The  upper  limit  of  20  cm  was  chosen 
based  on  the  author’s  experience  -  in  general  measurements  taken  from  distances  further 
than  this  are  excluded  from  scientific  analyses.  In  each  case,  it  is  assumed  that  the  tip  of 
the  fiber  is  pointed  at  the  center  of  patch  A.  Results  vary  little  by  fiber  size,  therefore  they 
are  only  shown  for  the  100-micron  fiber  in  the  rest  of  the  paper.  In  Figure  3.4b  the 
similarity  of  the  “contaminated”  spectra  of  patches  A  and  B  relative  to  their 
“uncontaminated”  versions  (from  Figure  3.3b)  are  shown  using  SAM  (Eq.  3.7),  for  both 
measurement  angles.  The  simulated  patches  are  squares  of  4.1  cm,  with  a  black  boundary 
separating  them,  and  in  both  the  conspicuous  and  camouflaged  case,  the  signal  remains 
uncontaminated  up  to  a  measurement  distance  of  about  10  cm.  Beyond  that,  the  cross- 
sectional  sampling  area  expands  into  neighboring  patches  and  there  are  two  points  to 
note.  First,  for  a  given  measurement  distance  the  degree  of  signal  contamination  depends 
on  the  measurement  angle;  the  45°  measurement  has  higher  SAM  values  (i.e.,  more 
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dissimilarity)  then  the  90°  measurement  at  any  given  distanee.  Second,  the  degree  of 
contamination  of  spectra  is  not  just  a  function  of  d  and  a,  but  also  of  the  neighboring 
colors.  Patches  A  and  B  have  different  neighboring  patches.  The  reflectance  spectrum  of 
patch  B  is  more  similar  to  the  spectra  of  the  patches  that  happen  to  be  immediately 
adjacent  to  it  than  patch  A  is  to  its  neighbors.  This  is  purely  a  result  of  the  arrangement  of 
color  patches  on  the  synthetic  test  target  and  affects  the  overall  SAM  scores  for  the  self¬ 
similarity  of  patches  A  and  B. 


a  =  90°,  d  =  5cm  a  -90",  d  -IOcm  a  =  90”.  d=15cm  a  =  90”,  d  =  20  cm 


(a) 


distance  (cm)  distance  (cm) 

(b)  (c) 

Figure  3.5  (a)  Cross-section  of  the  cone  of  acceptance  when  the  fiber  is  held  at  an  angle 
perpendicular  to  the  surface,  at  measurement  distances  5, 10, 15  and  20  cm  (the  dimension 
of  each  square  is  4.1  cm).  The  signals  measured  from  patches  A  and  B  are  expected  to 
remain  pure  up  to  d  =  10  cm.  (h)  In  the  conspicuous  case,  patches  A  and  B  have  little 
spectral  overlap,  and  that  translates  to  a  high  color  contrast  (AS)  value  for  both  the  frog  and 
the  ferret.  Beyond  d  =  10cm,  the  color  contrast  decreases  (the  signals  become  more  similar) 
with  both  distance  and  measurement  angle,  (c)  In  the  camouflaged  case,  the  spectrum  of 
patch  A  is  copied  to  the  location  of  patch  B.  The  color  contrast  is  zero  until  d  =  10  cm  and 
after  that,  the  spectra  quickly  get  contaminated.  The  signals  measured  at  a=90°  remain 
slightly  more  pure  than  at  a=45°.  The  dashed  line  indicates  the  just  noticeable  difference 
(JND)  threshold  of  1. 


The  same  analysis  is  repeated  with  the  spectra  in  patch  B  replaced  with  a  copy  of 
patch  A  to  simulate  an  example  where  an  animal  might  be  trying  to  match  the  colors  of 
its  background  to  fool  predators.  Originally,  patch  A  had  little  spectral  similarity  to  patch 
B.  Now,  patch  A  has  a  neighbor  with  an  identical  spectrum,  and  its  overall  level  of 
contamination  falls  compared  to  that  in  the  conspicuous  case.  The  new  patch  B  (blue), 
however,  has  less  spectral  overlap  with  its  neighbors  than  did  the  old  patch  B  (red,  case 
1),  and  its  overall  level  of  contamination  increases.  This  demonstrates  that  the 
measurement  distance  and  angle  alone  are  not  enough  to  predict  the  level  of 
contamination  of  a  color  signal  recorded  with  a  certain  optical  fiber,  and  that  the 
neighboring  colors,  which  are  often  difficult  to  predict  in  a  biological  application,  also 
play  a  role. 
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3.5.2  Perceptual  similarity  of  spectra 

Case  (1):  Conspicuous  animal 

Patches  A  and  B  were  ehosen  to  represent  the  pattern  of  a  eonspicuous  animal 
beeause  they  have  little  speetral  overlap.  The  reeeptor  noise  model  would  be  expeeted  to 
prediet  a  high  eolor  eontrast  between  these  patehes  in  the  eyes  of  both  the  diehromatie 
ferret  and  the  trichromatic  frog.  Figure  3.5a  shows  the  cross  sections  of  the  eone  of 
aeoeptanee  for  the  measurement  of  patehes  A  and  B,  at  distanees  d  =  S,  10, 15  and 
20  cm.  Beyond  d  =  10  cm,  the  disks  outgrow  the  square  patehes  and  we  expeet  the 
speetral  eontent  of  eaeh  pateh  to  beeome  eontaminated  with  speetra  from  neighboring 
patches.  This  in  turn  affeets  the  eolor  eontrast  between  the  patehes.  This  is  indeed  the 
case  as  seen  in  Figure  3.5b;  the  eolor  eontrast  (A5)  values  for  both  animals  are  well  above 
the  just  notieeable  differenee  (JND)  threshold  of  1  (indieated  by  the  dashed  horizontal 
line)  until  d  =  10  cm,  but  the  two  spectra  become  inereasingly  similar  (for  both  the  frog 
and  the  ferret)  after  that  point.  If  these  measurements  were  taken  at  a  distance  of  15  em, 
the  data  eould  have  led  to  an  ineorreet  eonelusion  that  the  blue  and  red  eolors  were 
indistinguishable  (JND<1)  to  the  ferret.  Note  that  the  A5  for  the  frog  never  falls  below 
the  JND  =  1  line,  meaning  that  these  two  patehes,  despite  their  speetral  eontamination, 
remain  distinguishable  to  the  triehromatie  animal,  whieh  has  the  advantage  of  having  an 
extra  photoreeeptor  eompared  to  the  diehromate. 

Case  (2):  Camouflaged  auimal 

For  the  ease  of  a  perfeetly  eamouflaged  animal,  patehes  A  and  B  have  identieal 
reflectanee  spectra,  but  they  have  different  neighboring  eolors  (see  Figure  3.4b  inset).  As 
expeeted,  the  speetra  of  patehes  remain  identieal  when  the  measurements  are 
uneontaminated,  up  to  J  =  10  cm,  and  for  both  animals  the  eolor  eontrast  is  zero  (Figure 
3.5e).  Beyond  that  point,  the  contrast  between  speetra  measured  at  a  =  90°  remains 
lower  for  a  slightly  higher  d  value  than  it  does  for  a  =  45°.  Note  that  for  this  example, 
the  magnitude  of  eolor  eontrast  for  both  visual  systems  are  comparable  implying  that  the 
measurements  of  pateh  speetra  (even  though  they  were  identieal)  are  eontaminated 
enough  beyond  d  =  11  cm  that  the  eolors  beeome  distinguishable  (JND>1)  to  both  the 
ferret  and  the  frog. 
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3.6  Discussion 

Observations  made  in  situ  are  key  elements  for  understanding  predator-prey 
interaetions  in  the  eontext  of  animal  eoloration  and  eamouflage  beeause  our  abilities  to 
reproduee  field  eonditions  in  the  lab  are  limited.  Thus,  speetral  data  eolleeted  in  situ  from 
freely  behaving  animals  in  the  wild  and  their  habitat  substrates  eontributes  signifieantly 
towards  our  interpretation  of  eamouflage  patterns.  However,  these  data  are  subjeet  to 
quality  issues  not  eneountered  in  the  lab.  In  laboratory  setups,  speetroseopie 
measurements  are  made  through  optieal  benehes,  whieh  often  eontain  vises  and  fiber 
holders  fixing  the  positions  of  the  sample  and  the  measurement  probes  for  eonsistent  and 
aeeurate  measurements.  For  field-based  studies,  standardizing  data  eolleetion  distanee 
and  angle  is  often  impossible.  Thus,  speetral  measurements  of  the  same  sample  taken  in 
the  lab  and  in  the  field  eould  be  different  beeause  of  variations  in  the  measurement 
distanee  and  angle,  whieh  depends  on  how  elose  the  surveyor  ean  get  to  the  animal  whose 
eolors  are  being  measured.  The  equations  presented  in  this  ehapter  aim  to  provide  a 
guideline  for  researehers  so  measurement  errors  made  in  the  field  eould  be  quantified. 

The  use  of  these  equations  are  not  limited  to  the  field  of  animal  eoloration  and 
eamouflage  sinee  the  ellipse  representing  the  interseetion  of  the  aeeeptanee  eone  and  a 
surfaee  only  depends  on  fiber  diameter  and  numerieal  aperture,  whieh  are  provided  by 
manufaeturers,  this  method  is  generalizable  to  applieations  of  speetroseopy  in  other 
fields. 

3.7  Conclusion 

Signal  eontamination  varies  based  on  the  speetral  eontent  of  the  neighboring 
eolors  of  a  pateh  in  addition  to  the  measurement  distanee.  Measurement  distanee  was 
more  important  than  measurement  angle  for  the  two  angles  eonsidered  (45°  and  90°). 

The  measurement  angle  simulated  (45  versus  90  degrees)  does  not  appear  to  have  a  large 
effeet  on  eolor  measurement  but  90°  may  be  preferable  if  the  shape  being  measured  has  a 
1:1  aspeet  ratio.  Fiber  size,  at  least  in  the  range  of  100-1000  mierometers,  has  a  negligible 
effeet  on  the  field  of  view  of  the  fiber.  The  patehes  used  in  this  example  were  squares  of 
4.1  em,  and  for  a  feature  of  that  size  and  shape,  the  maximum  measurement  distanee  that 
produeed  pure  eolors  was  approximately  10  em.  Most  features  measured  in  the  eontext  of 
animal  eoloration  are  smaller  than  4.1  em;  for  example  the  poison  frog  is  only  1-2  em  in 
size  (Siddiqi  et  al.  2004)  and  the  largest  body  eomponent  of  a  European  euttlefish  of 
mantle  length  20  em  is  around  3  em  (Akkaynak  et  al.  2013).  Animals  often  have  splotehy 
patterns  with  irregular  shapes  and  non-Lambertian  skins,  and  even  though  it  may  be 
possible  to  know  the  size  of  a  feature  that  will  be  measured,  there  may  be  no  a  priori 
information  regarding  its  neighboring  eolors.  Equations  3. 2-3. 6  presented  in  this  work 
eould  be  used  as  guidelines  before  speetral  measurements  from  animals  or  their  habitats 
are  made. 
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Chapter  4 


Quantification  of  cuttiefish  camoufiage  (S.  officinaiis):  a  study  of 
coior  and  iuminance  using  in  situ  spectrometry^ 

4.1  Introduction 

Coleoid  cephalopods  are  unrivaled  in  the  animal  kingdom  in  their  ability  to 
quiekly  and  dynamieally  ehange  their  body  patterns  for  signaling  and  eamouflage  (e.g., 
(Hanlon  and  Messenger  1988;  Hanlon  and  Messenger  1996).  Pattern  and  eolor  ehange 
are  aehieved  through  the  physiologieal  eontrol  of  ehromatophore  organs  (Florey  1969; 
Messenger  2001a),  and  struetural  refleetors  (iridophores  and  leueophores,  (Mathger  et  al. 
2009;  Wardill  et  al.  2012).  Many  laboratory  and  field  studies  have  shown  that 
eamouflage  behavior  in  euttlefish  is  visually  driven  (e.g.  (Chiao  and  Hanlon  2001b,  a; 
Chiao  et  al.  2005;  Barbosa  et  al.  2008a;  Barbosa  et  al.  2008b;  Barbosa  et  al.  2007;  Chiao 
et  al.  2007;  Kelman  et  al.  2007;  Allen  et  al.  2009;  Chiao  et  al.  2009;  Zylinski  et  al.  2009c, 
b,  a;  Chiao  et  al.  2010).  Counterintuitively,  these  animals  adaptively  tune  the  colors  of 
their  body  patterns  in  response  to  their  visual  backgrounds  without  the  use  of  color  vision 
(Marshall  and  Messenger  1996;  Mathger  et  al.  2006).  It  is  natural  to  wonder  how  well 
cuttlefish  can  match  the  colors  of  substrates  in  their  natural  habitats  if  they  cannot  sense 
color  with  their  eyes.  Our  observations  in  the  laboratory  and  in  the  wild  suggest  cuttlefish 
color  match  is  good  enough  to  fool  the  human  eye;  but  does  it  also  fool  the  visual 
systems  of  predators? 

In  this  work,  we  investigated  how  well  the  colors  of  cuttlefish  skin  matched  the 
colors  of  substrates  found  in  their  natural  habitats.  Specifically,  we  quantified  whether 
colors  from  cuttlefish  camouflage  patterns  were  discriminable  from  those  of  the 
background  substrates  in  the  eyes  of  predators.  To  do  so,  we  developed  a  systematic 
analytical  tool  to  compare  the  luminance  and  color  contrast  between  animal  and  substrate 
spectra  in  the  eyes  of  hypothetical  di-  and  trichromatic  fish  predators  and  supplemented 
our  spectral  comparison  by  further  analyzing  cuttlefish  color  matching  in  human  color 
space  using  the  International  Commission  on  Illumination  (CIE)  1931  XYZ  model 
(Wyszecki  and  Stiles  2000).  In  addition,  we  also  used  an  auxiliary  methodology  because 
sometimes  it  may  be  necessary  to  assess  the  similarity  of  two  spectra  independently  of 
the  perception  of  a  particular  visual  system.  For  example,  there  may  not  be  enough 
information  about  the  photoreceptor  types  or  ratios  of  a  certain  predator,  or  a  result  that 
can  be  generalized  across  visual  systems  may  be  desired.  In  such  cases,  purely 
mathematical  measures  of  spectral  similarity  can  provide  a  rough  approximation.  For  this 
reason,  we  used  Spectral  Angle  Mapper  (SAM),  a  technique  commonly  used  to  assess 
spectral  shape  similarity  in  the  field  of  remote  sensing  (Yuhas  et  al.  1992),  to  quantify  the 
similarity  of  cuttlefish  and  substrate  spectra.  We  found  that  SAM  correlated  moderately 
with  our  biological  measures  of  color  contrast,  making  it  comparable  to  a  traditional 
chroma  and  hue  based  measure  of  spectral  shape  similarity  (Endler  1990).  This  study  is 
the  first  to  collect  in  situ  spectral  data  for  color  match  assessment  of  European  cuttlefish 
{S.  officinalis). 
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Parts  of  this  chapter  was  previously  published.  See  (Akkaynak  et  al.  2013)  for  full 
reference. 
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4.2  Background  and  related  work 

Until  now,  spectrophotometric 
quantification  of  cuttlefish  camouflage 
had  only  been  performed  using  data 
collected  in  the  laboratory  (Mathger  et 
al.  2008;Chiao  et  al.  2011).  Mathger 
and  colleagues  (2008)  were  to  first  to 
quantitatively  analyze  the  color 
matching  abilities  of  cuttlefish.  They 
measured  reflectanee  speetra  from  10 
random  points  on  animals  (N  =  8)  and 
background  substrates  in  the 
laboratory  and  used  normalized 
Euclidean  distance  to  quantify  spectral 
difference.  They  found  that  cuttlefish 
showed  a  better  spectral  match  to  the 
1 1  natural  substrates  tested,  than  they 
did  for  the  2  artificial  substrates,  but 
did  not  analyze  the  discriminability  of 
colors  according  to  the  visual  systems 
of  predators.  Chiao  and  colleagues 
(2011)  also  performed  a  color  match 
assessment  in  the  laboratory,  but  they 
used  a  hyper-speetral  imager  to  image 
body  patterns.  In  addition,  they 
modeled  the  visual  systems  of 
hypothetical  predators  and  found  that 
the  eolors  cuttlefish  produced  were 
indistinguishable  in  the  eyes  of 
predators.  Both  studies  found  that 
cuttlefish  colors  showed  a  good 
mathematieal  and  perceptual  match  to 
the  substrates  tested  in  the  laboratory. 

However,  substrates  found  in  the 
natural  habitats  of  cuttlefish  show  a 
more  variation  in  terms  of  color  and 
structure  than  those  tested  in  the 
laboratory.  Therefore,  in  situ 
measurements  of  cuttlefish  skin  radiance  spectra  under  natural  illumination  and  in  natural 
habitats  are  erueial  to  provide  empirical  evidence  to  assess  the  degree  of  eolor  matehing. 

In  this  study,  we  colleeted  radianee  spectra  from  the  common  European  cuttlefish. 
Sepia  officinalis,  and  their  visual  surrounds  (sand,  gravel,  pebbles,  algae-covered  roeks, 
peacock’s  tail  alga,  hydrozoans.  Pinna  mollusc  shells,  etc.,  hereafter,  “substrates”) 
underwater  on  the  Aegean  Coast  of  Turkey.  We  describe  the  details  of  data  collection, 
calibration  and  analysis  next. 
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Figure  4.1  (a)  Spectral  data  were  collected  by  two  divers 
(DA  and  JJA).  One  diver  operated  the  spectrometer 
while  the  other  took  still  photographs  and  video  to 
document  the  sequence  of  measurements.  All 
measurements  were  taken  under  natural  daylight  after 
the  cuttlefish  had  habituated  to  the  presence  of  divers. 

(b)  Cuttlefish  body  pattern  components  measured  in  this 
study,  numbered  and  capitalized  according  to  their 
description  in  Hanlon  and  Messenger  (1988).  1  =  white 
posterior  triangle;  2  =  white  square;  3  =  white  mantle 
bar;  13  =  white  head  bar;  14  =  white  arm  triangle;  17  = 
anterior  transverse  mantle  line;  18  =  posterior 
transverse  mantle  line;  19  =  anterior  mantle  bar;  20  = 
posterior  mantle  bar;  21  =  paired  mantle  spots;  22  = 
median  mantle  stripe;  29  =  anterior  head  bar;  39  =  white 
square  papillae. 
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4.3  Methods 

Study  site,  animal  &  substrate  measurements 

Seven  cuttlefish  (mantle  lengths  between  15  and  22cm)  were  studied  off  the 
village  of  ^e§mealti  (Izmir)  on  the  Aegean  coast  of  Turkey  in  spring,  2011.  Fifteen 
radiance  datasets  were  collected;  each  data  set  consisted  of  one  animal,  one  location  and 
one  body  pattern.  Radiance  measurements  were  taken  using  a  USB2000  spectrometer 
(sensitivity  range:  200-850nm;  Ocean  Optics,  Dunedin,  FL)  coupled  with  a  Compaq  iPaQ 
handheld  computer  in  a  custom  underwater  housing  (Wills  Camera  Housings,  Victoria, 
Australia).  Optical  fibers  (50  and  100-micron  diameter)  were  used  to  collect  data  from 
cuttlefish  and  surrounding  substrates  {Figure  4.1a).  We  used  a  Spectralon  (Labsphere, 
UK)  target  as  a  white  reflectance  standard.  A  CCS  cosine  corrector  (Ocean  Optics, 
Dunedin,  FL)  was  attached  to  the  optical  fiber  for  collection  of  irradiance  data  and  an  LS- 
1-CAL  tungsten  light  source  (Ocean  Optics)  was  used  to  obtain  absolute  intensity  values. 
This  calibration  was  done  in  a  dark  room  to  minimize  stray  light.  Our  equipment  design 
is  documented  in  (Roelfsema  et  al.  2006)  and  has  been  used  for  fieldwork  by  many  others 
(Leiper  et  al.  2012;  Hedley  et  al.  2012;  Lim  et  al.  2009;  Cheney  et  al.  2008;  Matz  et  al. 
2006;  Cheney  and  Marshall  2009). 

Photographs  were  taken  with  a  24-70  mm  lens  on  a  Canon  EOS  1-Ds  Mark  II 
digital  camera  in  a  Subal  housing  equipped  with  a  dome  port.  Videos  recorded  with  a 
compact  FlipCam  (Irvine,  CA)  documented  the  underwater  data  collection. 

Measurements  were  taken  under  natural  light,  at  depths  shallower  than  5  meters.  Animals 
were  carefully  approached  by  two  divers  (DA  &  JJA)  until  they  habituated  to  the  divers’ 
presence  and  did  not  show  any  signaling  behavior  (e.g.,  unilaterally  expressed  pattern 
components  or  Paired  Mantle  Spots  as  part  of  a  Deimatic  display;  see  (Hanlon  and 
Messenger  1988). 

Animals  were  allowed  to  settle  in  a  location  of  their  choice  and  substrates  were 
not  altered,  with  one  exception:  before  data  acquisition,  white  rocks  were  placed  near  one 
cuttlefish  to  evoke  a  weak  pattern  (Figure  4.2g).  This  pattern  is  known  to  be  the 
camouflage  response  to  white  cues  in  a  cuttlefish’s  visual  surrounds  {S.  pharaonis, 

(Chiao  and  Hanlon  2001a);  S. officinalis,  e.g.,  (Barbosa  et  al.  2007;  Mathger  et  al.  2007). 

In  each  case,  ten  to  fifteen  radiance  measurements  were  recorded  for  each  body 
component  (Figure  4.2b),  background  element,  or  irradiance  measurement  and  their 
average  was  used  for  the  data  analysis.  If  the  cuttlefish  moved  or  changed  body  patterns 
during  data  acquisition,  the  animal  was  allowed  to  re-habituate  and  measurements  from 
the  new  location/body  pattern  were  recorded  as  a  new  data  set.  Similarly,  if  lighting 
conditions  changed  significantly  (e.g.,  in  response  to  passing  clouds),  a  new  set  of 
irradiance  and  white  standard  measurements  was  recorded.  In  both  cases,  we  only 
analyzed  datasets  that  were  taken  under  consistent  lighting  conditions.  In  all  cases,  the 
probe  head  of  optical  fiber  was  less  than  3  cm  away  from  the  feature  whose  spectrum  was 
being  recorded  (see  Chapter  3).  We  ensured  that  the  diver’s  hand  did  not  shadow  the  area 
of  interest. 
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Figure  4.2  (a-g)  Presentation  of  seven  datasets;  in  (i)  we  present  a  photograph  of  the  scene,  an  outline  of  the 
scene  showing  the  spots  radiance  spectra  were  recorded  from  and  normalized  radiance  spectra  we 
measured;  in  (ii)  the  Spectral  Angle  Mapper  score  computed  between  the  spectra  of  each  animal  component 
and  background  substrate  are  shown.  For  example,  for  the  animal  in  (a),  the  bars  represent  the  SAM  score 
between  pairs  of  features:  (“a”)  &  (“1”),  (“a”)  &  (“2”),  (“a”)  &  (“3”),  (“a”)  &  (“19”),  (“a”)  &  (“20”),  (“b”) 

&  (“1”),  (“b”)  &  (“2”),  (“b”)  &  (“3”),  (“b”)  &  (“19”),  (“b”)  &  (“20”)  and  so  on.  They  are  sorted  in 
descending  order.  See  text  for  details.  We  repeat  here  the  pattern  names  we  have  used  in  text  for  each  photo: 
(a)  uniform/stipple,  (b)  dark  uniform  with  weak  zebra  stripes,  (c)  mottle/weak  weak,  (d)  mottle,  (e)  mottle, 

(f)  weak  zebra,  and  (g)  weak.  See  supplementary  online  resource  2  for  larger,  high  resolution  images. 
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Assessment  of  chromatic  similarity  between  animal  and  background  spectra  using  a  non- 
biological  measure  (Spectral  Angle  Mapper) 

In  the  field  of  remote  sensing,  automated  speetral  library  seareh  algorithms 
developed  for  hyper-speetral  images  (Chang  2003;  Sweet  2003;  Freek  2006;  Nidamanuri 
and  Zbell  2011)  are  used  to  eompare  refieetanee  speetra  of  known  targets  to  those  of 
novel  speetra  by  eomputing  a  sealar  similarity  seore  between  them.  For  these  algorithms, 
stoehastie  methods  are  more  frequently  used  than  deterministie  methods  beeause  imaging 
eonditions  ean  be  imperfeet  and  beeause  the  high  speetral  resolution  of  a  hyper-speetral 
sensor  often  results  in  more  than  one  material  speetral  signature  in  a  given  pixel.  In  our 
study,  speetral  data  were  point-by-point  measurements  of  solid  eolor  patehes  where  the 
area  of  the  pateh  was  mueh  wider  than  the  diameter  of  the  speetrometer  fiber.  Henee,  we 
use  a  deterministie  method,  Speetral  Angle  Mapper  (SAM),  from  the  field  of  hyper- 
speetral  image  elassifieation  in  our  assessment  of  speetral  shape  similarity.  SAM  is  the 
most  eommonly  used  speetral  angle-based  similarity  measure  ((Yuhas  et  al.  1992); 

(Kruse  et  al.  1993)  and  it  is  the  normalized  inner  produet  of  two  vectors.  It  is  computed 
as  follows: 


e  = 


cos 


(  \ 
vii^iiiii^zii; 


(4.1) 


where  “Si”  and  “S2”  are  the  two  spectra  vectors  being  compared.  Each  continuous 
radiance  spectrum  is  vectorized  (denoted)  into  31  dimensions  by  taking  its  value  every  10 
nm  in  the  visible  range  (400-700nm).  “T”  denotes  the  transpose  of  a  matrix  and  the  ||.|| 
symbol  denotes  Euclidian  norm;  the  division  by  the  vector  norms  indicates  that  SAM  is 
indifferent  to  the  magnitude  of  the  vectors  (brightness)  and  only  calculates  similarity  of 
spectral  shape  (color).  A  small  angle  between  two  vectors  indicates  that  the  spectra  are 
similar  in  shape. 

Calculation  of  color  difference  using  chroma  and  hue 

A  common  way  to  assess  the  similarity  of  two  spectra  independent  of  a  visual 
system  is  to  compute  their  chroma  (C),  hue  (H)  and  brightness  (B)  values,  and  calculate 
the  Euclidean  distance  between  them  (D).  A  small  Euclidean  distance  means  that  the  two 
colors  are  similar.  Here,  we  used  the  segment  classification  analysis  of  spectra  from 
(Endler  1990)  to  calculate  hue  and  chroma.  Our  goal  is  to  compare  the  performance  of 
this  method  to  that  of  SAM.  SAM  metric  does  not  take  brightness  into  account  and  for  a 
fair  comparison,  before  chroma  and  hue  values  are  calculated  each  spectra  should  be 
multiplied  with  a  constant  to  equalize  their  overall  brightness  (see  (Endler  1990)  for 
further  details),  hollowing  this,  the  distance  between  the  two  colors  are  found  from: 

D  =  V(i/i  -  H2Y  +  (Cl  -  C2Y  +  (Bi  -  B2Y  (4.2) 


where  =  B2  due  to  the  brightness  equalization  step  carried  out. 


53 


Assessment  of  spectral  similarity  between  animal  and  background  in  the  eyes  of 
hypothetical  di-and  trichromatic  fish  predators:  luminance  and  color  contrast 

Although  the  details  of  visual  systems  of  cuttlefish  predators  are  not  known 
{Serranus  cabrilla  is  the  only  fish  species  observed  directly  to  prey  on  S.  officinalis  in  the 
Mediterranean  sea),  we  may  speculate  that  cuttlefish  are  preyed  upon  by  a  variety  of 
vertebrate  and  invertebrate  predators  with  different  visual  systems  ((Hanlon  and 
Messenger  1988;  Hanlon  and  Messenger  1996).  Thus,  we  chose  one  dichromatic  fish  and 
one  trichromatic  fish  as  their  potential  predators  to  simulate  their  views  of  these 
camouflaged  cuttlefish.  In  dichromatic  fish,  the  kmax  of  S  and  M  cones  was  460  and  570 
nm.  In  trichromatic  fish,  the  k^ax  of  S,  M,  and  L  cones  was  460,  540,  and  570  nm.  N.B., 
although  the  choice  of  these  kmax  of  dichromatic  and  trichromatic  cones  was  arbitrary, 
shifting  the  k^ax  of  these  cones  up  or  down  10-20  nm  did  not  visibly  affect  the  results. 

We  carried  out  this  analysis  for  photoreceptor  ratios  1:1:1  and  1:2:2  for 
trichromats  and  1 : 1  and  1 :2  for  dichromats;  these  are  typical  fish  retina  cone  mosaic 
patterns  (Shand  et  al.  1999;  Cheney  and  Marshall  2009).  The  results  of  our  analysis  in 
both  cases  were  similar,  and  we  only  present  results  for  the  ratios  1:1:1  for  trichromats 
and  1:1  for  dichromats. 

Luminance  contrast  (AL)  is  calculated  as  —  L,2,  where  L  =  ln(/c(2);  k  is  the 
von-Kries  adaptation  coefficient  and  Q  is  the  quantum  catch  value.  Spectra  were 
collected  under  the  same  ambient  conditions  for  each  dataset,  therefore  the  same  von- 
Kries  coefficient  was  applied  to  both  luminance  values  (/ci  =  k2).  Thus,  AL  = 

In  =  In  — .  Quantum  catch  value  is  computed  as  follows: 

^2  <22  Q2 


Qi^ 


Jr^max 

Amin 


i?(A)/(A)5i(A)dA 


(4.3) 


where  i?(l)is  the  reflectance  spectrum  of  the  feature  of  interest,  7(1)  is  the  spectrum  of 
the  light  source,  5’j(l)  is  the  spectral  sensitivity  of  the  photoreceptor  (i  =  S,M,L), 

A^jjiand  ^re  the  bounds  of  the  visible  spectrum,  respectively.  5’j(l)  was  based  on 
the  visual  pigment  template  of  Stavenga  et  al.  (1993)  with  beta-peak  shifted  linearly,  after 
Palacios  et  al.  (1996)  using  the  equation  generated  by  Hart  (1998).  For  calculating 
luminance  contrast,  we  only  used  the  long  wavelength  receptors  since  they  are  thought  to 
be  responsible  for  luminance  contrast  in  fish  (Marshall  et  al.  2003a).  We  computed 
luminance  contrast  between  the  cuttlefish  body  parts,  as  well  as  between  cuttlefish  parts 
and  background  substrates.  For  color  contrast,  we  used  the  metric  AS  (with  Weber 
fraction  0.05)  as  defined  by  the  Vorobyev-Osorio  Receptor  Noise  Model  (Vorobyev  and 
Osorio  1998). 


54 


dichromat 


trichromat 


Figure  4.3  (a-g)  In  (i),  luminance  contrast  is  shown  for  two  cases:  (1)  amongst  the  spectra  of  the  cuttlefish 
body  components;  (2)  between  the  spectra  cuttlefish  components  and  the  background  substrates.  In  (ii), 
color  contrast  calculated  between  spectra  of  cuttlefish  components  and  substrates  are  presented  for 
hypothetical  di  and  tri-chromats  we  modelled.  Luminance  contrast  values  shown  are  for  trichromats  only 
and  are  identical  for  dichromats.  See  text  for  details.  In  all  plots,  the  green  line  indicates  a  “just  noticable 
difference”  of  1;  pairs  of  features  that  fall  below  this  value  cannot  be  distinguished  by  the  visual  system 
under  consideration. 
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Visualization  of  spectral  similarity  between  cuttlefish  and  backgrounds  in  human  and 
hypothetical  fish  predator  color  spaces 

It  is  helpful  to  visualize  the  speetral  mateh  between  a  euttlefish  and  its 
surroundings  as  a  human  would  have  seen  it.  We  used  the  CIE  1931  (2  degree  observer) 
eolor  matehing  funetions  to  obtain  XYZ  tri-stimulus  values  from  radianee  speetra.  We 
then  plotted  the  xy-eoordinates  of  the  tri-stimulus  values  of  cuttlefish  body  components 
and  substrates  on  a  CIE  1931  Chromaticity  Diagram;  see  (Wyszecki  and  Stiles  2000)  for 
details.  Eor  comparison,  we  included  the  xy-coordinates  corresponding  to  the  radiance 
spectra  of  the  24  patches  of  a  Macbeth  ColorChecker  (Xrite,  Inc.  Grand  Rapids,  MI), 
assumed  to  be  viewed  under  a  CIE  D65  illuminant  (noon  daylight,  6504K).  Patches  of  a 
Macbeth  ColorChecker  are  designed  to  represent  colors  humans  encounter  on  a  daily 
basis.  N.B.;  the  CIE  XYZ  chromaticity  diagram  is  not  perceptually  uniform;  therefore, 
the  same  distance  between  two  pairs  of  colors  on  the  XYZ  diagram  may  correspond  to 
different  perceptual  distances.  The  XYZ  tri-stimulus  values  cannot  be  visualized  directly; 
to  offer  a  visual  comparison  of  the  variety  of  cuttlefish  and  habitat  colors  at  our  field  site, 
we  converted  the  XYZ  tri-stimulus  values  into  the  sRGB  space  and  created  color  patch 
assemblies  for  both  animal  and  substrate  data.  Eor  details  on  converting  XYZ  tri-stimulus 
values  to  RGB  color  spaces,  see  (Reinhard  et  al.  2008)). 

Chromaticity  diagrams  are  also  used  in  the  studies  of  animal  color  vision  (see 
(Pike  201  la)  and  (Kelber  et  al.  2003)  as  graphical  representations  of  perceived  colors. 

We  followed  the  methodology  described  in  (Kelber  et  al.  2003)  to  plot  the  loci  of  colors 
measured  from  cuttlefish  and  surrounding  substrate  on  the  Maxwell  triangle  for  a 
hypothetical  trichromatic  observer  with  kmax  values  for  S,  M,  and  L  cones  460,  540,  and 
570  nm. 

Simulation  of  color  and  luminance  contrasts  at  depth 

Underwater,  the  available  light  field  changes  in  intensity  and  spectral 
composition,  changing  the  appearance  of  objects  in  response  to  factors  such  as:  depth, 
time  of  day,  weather  conditions  and  the  amount  of  suspended  particles  in  the  water 
column.  To  illustrate  the  effect  of  this  change  on  camouflage,  data  from  a  uniform  dark 
animal  collected  at  I  m  depth  (an  animal  that  did  not  appear  well  color  matched  to  its 
surroundings)  were  used  to  simulate  the  appearance  of  its  colors  at  a  depth  of  10  m  using 
irradiance  spectra  collected  at  10  m  at  our  study  site.  The  simulation  was  done  as  follows. 
Eirst,  the  radiance  spectra  from  the  original  dataset  were  converted  into  reflectance  by 
dividing  the  difference  of  radiance  of  the  feature  of  interest  and  dark  noise  by  the 
difference  of  white  standard  radiance  and  dark  noise.  Second,  these  reflectance  spectra 
were  multiplied  with  irradiance  spectra  we  recorded  in  situ  at  10  meters  depth.  Third, 
simulated  animal  and  substrate  radiance  spectra  were  assessed  for  similarity  and 
discriminability  using  methodology  described  above.  Color  patches  were  simulated  in 
sRGB  space  using  the  tri-stimulus  values  obtained  from  the  CIE  1931  XYZ  model. 

4.4  Results 

In  Eigure  4.3i  radiance  measurements  taken  in  the  field  are  summarized. 
Measurements  taken  from  cuttlefish  components  are  labeled  on  the  outlines  of 
photographs  with  black  numbers,  those  taken  from  substrates  are  labeled  with  blue 
letters.  The  same  color  convention  is  used  in  spectral  curves  and  each  curve  is  normalized 
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by  its  maximum  value  to  emphasize  similarity  of  shape.  The  red  curve  in  each  plot  is  the 
(normalized)  spectrum  recorded  from  a  Spectralon  target  and  represents  the  shape  of 
“white”  under  ambient  light  conditions. 

Use  of  Spectral  Angle  Mapper  for  assessing  spectral  shape  similarity  between  animal 
and  background 

Figure  4.3ii  shows  the  Spectral  Angle  Mapper  scores  computed  between  each 
cuttlefish  component  and  background  substrate  measured.  The  limits  of  each  plot  were 
adjusted  to  the  minimum  and  maximum  values  of  SAM  encountered  across  all  datasets  (0 
and  0.4508,  respectively).  The  score  computed  by  SAM  is  a  measure  of  how  well  two 
multi-dimensional  vectors  are  aligned  and  does  not  carry  a  biological  meaning. 

Therefore,  while  the  SAM  scores  cannot  give  any  information  regarding  the 
discriminability  of  two  colors  from  the  perspective  of  any  visual  system,  they  are 
informative  about  how  similar  the  shapes  of  two  spectra  are,  which  can  be  compared  to 
chromaticity.  In  general,  the  lower  the  SAM  score,  the  more  similar  the  colors.  For 
example,  in  Figure  4.2a,  the  cuttlefish  has  a  uniform  body  pattern,  which  appears  well 
matched  to  the  surrounding  substrates  in  color.  The  corresponding  SAM  plot  has  low 
scores  throughout.  The  animal  in  Figure  4.2b  also  has  a  uniform  pattern  but,  unlike 
Figure  4.2a,  it  does  not  match  the  surrounding  sand.  This  difference  is  captured  in  the 
magnitude  of  the  values  displayed  in  the  SAM  plots. 

Assessment  of  luminance  and  color  contrasts  in  the  eyes  of  hypothetical  di-  and 
trichromatic  fish  predators 

In  Figure  4.3i,  we  present  luminance  contrast  among  cuttlefish  components  and 
luminance  contrast  between  cuttlefish  components  and  substrates.  Contrast  among 
cuttlefish  components  is  an  indicator  of  whether  the  cuttlefish  pattern  is  uniform  or  non- 
uniform  (i.e.  mottle,  zebra  or  weak),  and  contrast  between  animal  and  substrate  is  an 
indicator  of  how  similar  the  cuttlefish  components  are  from  the  substrate  in  brightness. 
Figure  4.3ii  shows  color  contrast  between  cuttlefish  and  substrates  as  seen  by 
hypothetical  di-  and  trichromats.  In  both  (i)  &  (ii),  a  value  of  1  is  a  “just  noticeable 
difference”  (JND)  and  is  marked  with  a  green  line. 

Pattern  1  (Figure  4,3a),  Luminance  contrast  values  below  the  JND  =  1  line  suggest  this 
animal  has  a  uniform  pattern,  and  matches  the  substrates  in  intensity.  The  photograph  in 
Fig.  2a  confirms  these  observations.  Low  color  contrast  values  suggest  that  this  animal  is 
difficult  for  both  di-  and  trichromats  to  distinguish  from  its  background. 

Pattern  2  (Figure  4,3b),  Low  luminance  contrast  among  animal  components  indicates  a 
uniform  pattern.  Almost  all  cases  of  luminance  contrast  between  animal  and  substrates 
are  above  the  visual  threshold,  indicating  this  animal  did  not  match  the  substrate  in 
luminance.  Color  contrast  values  suggest  this  animal  can  be  distinguished  from  its 
background  by  both  di-  and  trichromats,  but  note  that  color  contrast  values,  overall,  are 
lower  for  the  dichromat  (on  average,  ~  0.7)  indicating  that  it  is  harder  for 

the  dichromat  to  detect  this  animal  when  compared  to  a  trichromat.  Indeed,  there  was  a 
mismatch  between  this  cuttlefish  in  a  dark  uniform  body  pattern  (with  weak  zebra  stripes) 
and  the  sand  in  its  immediate  surroundings  as  confirmed  by  Figure  4.2b. 
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Pattern  3  (Figure  4,3  c).  In  this  dataset,  we  observed  that  euttlefish  eomponents  had  high 
and  low  luminance  contrast  values  among  themselves  indicative  of  a  non-uniform 
(mottle,  zebra  or  weak  pattern).  The  photograph  in  Figure  4.3c  shows  a  mottle/weak 
weak  body  pattern.  In  both  color  and  luminance  contrast,  when  compared  to  the  dark 
green  Posedonia  seagrass  (“b”),  all  cuttlefish  components  scored  above  the  visual 
threshold.  In  response  to  visual  cues  from  a  nearby  three-dimensional  structure  with 
projections  approximately  the  width  of  its  arms,  this  animal  raised  its  first  pair  of  arms,  a 
postural  component  of  cuttlefish  camouflage  (Barbosa  et  al.  2012).  Although  some  body 
components  matched  the  gravel  in  luminance  and  color  contrast,  the  arm  posture  and 
overall  body  pattern  suggested  this  animal  may  have  performed  masquerade  camouflage 
in  response  to  visual  cues  from  the  three  dimensional  Posedonia  seagrass  (for  discussions 
of  masquerade  camouflage,  see  (Stevens  and  Merilaita  2009;  Skelhom  et  al.  2010; 
Buresch  et  al.  2011;  Skelhorn  and  Ruxton  2011). 

Pattern  4  (Figure  4,3d).  Luminance  contrast  plots  suggest  this  animal  has  a  uniform 
pattern  and  matches  the  substrate  in  intensity.  The  photograph  in  Figure  4.3d  shows  that 
this  animal  had  a  uniform  body  pattern  with  some  aspects  of  a  mottle.  While  this  animal 
is  well  matched  to  its  surroundings  in  luminance,  its  overall  pattern  is  detectable  in  terms 
of  color  contrast  to  both  di-  and  trichromats  (Figure  4.3d,ii).  It  was  difficult  to  distinguish 
this  cuttlefish  from  nearby  3D  objects,  suggesting  this  animal  might  have  performed 
masquerade  camouflage  instead  of  background  matching  (e.g.,  (Buresch  et  al.  2011). 

Note  that  the  radiance  spectra  measured  from  the  components  of  this  animal  were  similar 
to  those  measured  from  a  dark  brown  hydrozoan  roughly  two  mantle  lengths  away 
(Figure  4.3d,  inset). 

Pattern  5  (Figure  4,3  e).  The  luminance  contrast  values  suggest  a  mottle,  zebra,  or  weak 
pattern  and  the  mostly  high  color  contrast  values  in  (ii)  suggest  this  animal  may  be  easy 
to  spot  against  the  background.  The  photo  shows  that  this  animal  had  a  mottle  pattern 
with  weak  expression  of  the  White  square  and  Median  mantle  stripes,  components  often 
seen  in  weak  body  patterns.  All  animal  body  components  closely  matched  the  nearby 
Pinna  mollusc  shell  (“a”)  in  luminance  contrast  but  had  high  contrast  when  compared  to 
the  gravel  (“b”).  Overall,  this  animal  could  be  detected  by  a  hypothetical  di-or  tri¬ 
chromatic  predator. 

Pattern  6  (Figure  4,3f).  The  low  values  of  luminance  contrast  among  animal  components 
suggest  a  uniform  pattern.  The  photograph  shows  that  the  animal  actually  had  a  weak 
zebra  pattern,  showing  low  overall  contrast.  This  animal  allowed  us  to  collect  data  from 
individual  light  and  dark  stripes  within  two  body  pattern  components:  the  Posterior 
mantle  bar  (“20”)  and  the  White  mantle  bar  (“3”).  As  in  the  case  of  pattern  3,  the 
cuttlefish  components  better  matched  the  green  Posedonia  seagrass  in  luminance  than  in 
color  contrast  (substrates  “a”  and  “b”).  This  result  is  intuitive  as  the  cuttlefish  is  not 
capable  of  producing  a  green  color  with  its  pigmented  chromatophores,  but  can  alter  the 
intensity  of  skin  to  appear  light  or  dark.  Overall,  this  animal  could  be  detected  by  a 
hypothetical  di-or  tri-chromatic  predator. 
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Pattern  7  (Figure  4,3g),  Luminance 
contrast  among  cuttlefish  components 
suggest  that  this  animal  has  a  uniform 
body  pattern;  however  from  Figure  4.2g, 
we  see  that  it  had  a  weak  pattern.  Its 
weak  pattern  was  turned  on  after  white 
rocks  were  placed  nearby.  When 
compared  among  cuttlefish  components 
and  substrates,  luminance  contrast 
values  were  high;  indicating  the  animal’s 
components  are  darker  than  the 
surrounding  white  rocks,  with  the 
exception  of  its  White  square.  The 
luminance  of  the  White  Square  (“2”) 
was  similar  to  the  luminance  of  the  other 
components.  The  cuttlefish  components 
were  most  similar  to  substrates  (“c”  and 
“d”)  in  luminance  contrast,  and  nearly 
indistinguishable  in  the  eyes  of  both  di- 
and  trichromats  in  terms  of  color 
contrast.  Note  that  the  White  square  was 
not  pure  white,  rather,  closer  to  a  light 
brown  color.  White  square  luminance 
and  color  are  modulated  by  pigmented 
brown,  orange  and  yellow 
chromatophore  organs  overlying  the 
structural  reflectors  (lecuophores) 
responsible  for  whiteness  in  this  animal 
(Mathger  et  al.  2009). 

Comparison  across  all  patterns 

Figure  4.4a  summarizes  the 
mean  JND  values  across  all  datasets.  In 
almost  all  cases,  the  dichromat  JND 
values  were  equal  to  or  smaller  than  the 
trichromat  JND  values,  suggesting  that  it 
was  more  difficult  for  the  dichromats  to 
distinguish  the  cuttlefish  components 
from  the  substrates.  Similarly,  the  mean 
luminance  contrast  of  the  cuttlefish  components,  when  compared  to  the  surrounding 
substrate,  was  less  than  or  slightly  higher  than  1  JND;  the  cuttlefish  generally  did  a  good 
job  matching  the  luminance  of  the  substrates.  For  datasets  “b”  and  “blOm”  (the  latter  is 
the  animal  whose  appearance  is  simulated  at  10  meters,  described  in  the  “Simulation  of 
spectra  at  depth”  subsection),  the  luminance  contrast  was  nearly  unchanged,  while  the 
mean  color  contrast  fell  to  the  JND  =  1  limit.  The  mean  values  in  dataset  “e”  are  likely  to 
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Spectral  Angle  Mapper  (SAM) 

Figure  4.4  (a)  Mean  just  noticable  difference  (JND) 
values  shown  for  each  dataset  and  (h)  the  percent  of 
cuttlefish  components  that  are  below  1  JND  for  each 
dataset,  (b)  We  found  that  SAM  has  moderate 
correlation  (0.5<|r|<0.7)  to  A5,  a  biological  measure  of 
color  contrast  for  the  data  we  present.  Note  that  variance 
in  the  predicted  color  conrtrast  score  increases  with 
increasing  spectral  angle  scores  (heteroscedasticity).  This 
level  of  correlation  did  not  change  when  we  varied  the 
photoreceptor  ratios,  and  shifted  peak  wavelengths  of 
S,M  and  L  cone  types  by  10-20  nm  for  the  visual  systems 
we  tested.  This  is  comparable  to  D,  a  chroma  and  hue 
based  spectral  similarity  metric  which  is  independent  of 
a  particular  observer’s  visual  system.  For  the  data  we 
present,  D  values  were  also  moderately  correlated  with 
A5.  Data  shown  above  for  A5  =  450,  =  530  and 

Xi  =  560  nm;  r5^jif,as  —  0. 618  and  ^5  =  0. 608  (not 
plotted). 
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be  affected  by  the  measurement  of  the  dark  eyespots;  these  spots  do  not  usually  appear  as 
a  part  of  a  camouflage  pattern  and  were  displayed  as  a  warning  to  the  divers. 

Figure  4.4b  shows  the  percentage  of  pairs  of  components  that  had  JND  values 
less  than  1 .  Animals  in  datasets  “a”  and  “c”  appear  to  have  the  best  overall  color  and 
luminance  match  in  the  eyes  of  both  di-  and  trichromatic  predators,  and  those  in  “d”,  “e” 
and  “f’  seem  to  have  done  a  better  job  matching  luminance  than  color.  Overall,  the 
animal  in  dataset  “b”  was  the  worst  color  matched  animal.  Its  simulated  appearance  at  10 
meters  depth  was  more  conspicuous  than  the  animal  at  1 -meter  depth. 

The  relationship  between  SAM,  chroma  and  hue  based  color  difference  (D),  and  color 
contrast  (AS) 

The  SAM  scores  computed  between  cuttlefish  components  and  substrates  for  all 
datasets  showed  moderate  correlation  to  the  color  contrast  values  computed  using  the 
(AS)  metric;  the  absolute  value  of  the  correlation  coefficient  varied  between  0.5  and  0.7 
(p  «  0)  for  all  cases  of  photoreceptor  ratios  we  tested  (Figure  4.4c).  Note  that 

the  data  appear  to  be  heteroscedastic.  D,  the  color  difference  computed  using  chroma  and 
hue  values,  also  correlated  moderately  with  AS.  When  compared  against  each  other 
however,  SAM  and  D  had  a  weak  a  correlation  (on  average,  p  «  0.4).  When  assessed 
individually,  on  average  SAM  was  moderately  correlated  with  hue,  and  weakly  correlated 
with  chroma. 

Visualization  of  spectral  similarity  between  cuttlefish  and  backgrounds  in  human  color 
space 

We  plotted  the  loci  of  xy  values  for  each  animal  component  and  substrate  on  a 
Chromaticity  Diagram  (Figure  4.5a).  Cuttlefish  body  components  and  substrates  were 
constrained  to  a  general  area  around  the  point  x=0.4  &  y=0.35  near  the  locus  of  gray 
colors  at  X  =  0.33,  y  =  0.33  (the  “white  point”  or  the  “achromatic  point”).  All  spectra 
measured  from  cuttlefish  body  pattern  components,  and  spectra,  were  converted  to  sRGB 
colors  and  are  shown  with  a  corresponding  plot  of  “white”  (a  highly  reflective  Spectralon 
white  standard)  under  ambient  light  conditions  (Figure  4.5b).  This  visualization  revealed 
that  the  colors  of  both  the  animals  and  substrates  at  our  study  site  were  limited  to  a 
narrow  range  within  the  gamut  of  colors  visible  to  humans. 

Simulation  of  spectra  at  depth 

One  cuttlefish  (see  Figure  4.2b;  data  taken  at  Im  depth)  poorly  matched  its 
surrounding  sandy  substrate.  We  used  the  irradiance  profde  we  recorded  at  our  study  site 
at  a  depth  of  10  meters  to  obtain  a  theoretical  ambient  light  field  and  simulate  this 
animal’s  radiance  spectra  had  they  been  collected  at  that  location  (see  Methods  for 
details).  Simulated  spectra,  SAM,  luminance  and  color  contrast  results  are  presented  in 
Figure  4.4a.  At  this  depth,  the  shapes  of  all  spectra  become  similar  in  shape  due  to  the 
ambient  light  conditions  as  shown  by  the  spectral  curves.  Overall,  the  SAM  scores 
decreased  by  about  50%;  indicating  that  the  spectra  became  more  similar  in  shape  at 
depth.  Luminance  contrast  remained  nearly  the  same.  This  is  expected,  as  luminance 
contrast  is  a  ratio  of  the  quantum  catches,  and  remains  unchanged  when  the  same  process 
attenuates  both  spectra.  The  color  contrast  plots  show  that  in  the  eyes  of  both  di-  and 
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trichromat,  the  color  match  at  10  m  depth  has  gotten  better,  and  it  is  now  harder  to 
distinguish  this  animal  from  its  background. 

In  Figure  4.6b  Maxwell  triangles  show  the  change  in  the  appearance  of  colors 
from  1  m  to  10m.  At  10  meters,  cuttlefish  and  substrate  colors  form  tighter  clusters  than 
they  did  at  1 -meter  depth.  The  sRGB  appearance  of  colors  shows  that  at  10  meters,  the 
cuttlefish  and  substrate  colors  are  indistinguishable. 

4.5  Discussion 


The  utility  of  in  situ  spectrometry  in  studying  animal  camouflage 

Here  we  presented  spectrophotometric  field  data  collected  from  camouflaged 
cuttlefish  {Sepia  officinalis)  and  some  of  their  surrounding  substrates.  We  have  advanced 
previous  studies  by  collecting  data  in  the  cuttlefish’s  natural  benthic,  near-shore 
environment  where  the  daylight  spectrum  is  affected  by  atmospheric  conditions,  water 
quality  and  depth  (Jerlov  1968,  Jerlov,  1976  #205;  Tyler  and  Smith  1970).  Studying 
cuttlefish  camouflage  under  natural  conditions  is  essential  for  the  study  of  animal  and 


Figure  4.5  Analysis  of  cuttlefish  color  matching  with  respect  to  the  human  visual 
system,  (a)  Chromaticity  plot  showing  the  loci  of  cuttlefish  and  substrate  xy 
chromaticity  coordinates  obtained  from  radiance  spectra  measured  in  the  field. 
Black  squares  show  the  xy  coordinates  of  the  patches  of  a  Macbeth  ColorChecker. 
(b)  Color  assemblies  of  all  cuttlefish  components  measured.  Each  color  patch  in  an 
assembly  is  accompanied  by  a  corresponding  white  patch,  representing  the  way 
“white”  looked  under  the  ambient  conditions.  Depending  on  depth,  time  of  day, 
visibility  etc.,  white  may  appear  as  shades  of  green  and  blue. 


background  luminance  and  color  because  it  is  under  these  eonditions  that  cuttlefish 
camouflage  body  patterns  have  evolved  to  suoeessfully  deceive  the  eyes  of  predators. 
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Recently,  field  spectrometry  data  collected  by  Hanlon  et  al.  (2013)  were  analyzed  for  a 
different  species,  the  giant  Australian  cuttlefish  {Sepia  apama),  using  a  similar  approach 
to  quantify  camouflage  in  the  eyes  of  potential  fish  predators.  Studies  assessing  the  color 
signals  of  animals  through  the  visual  systems  of  their  known  predators  have  been  done 
for  chameleons  (Stuart-Fox  and  Moussalli  2008),  spiders  (Heiling  et  al.  2003),  (Thery 
and  Casas  2002),  fish  (Marshall  and  Vorobyev  2003)  and  birds  (Endler  and  Mielke  Jr 
2005).  Spectrometers,  however,  are  not  ideal  instruments  for  the  assessment  of  overall 
animal  patterns  because  they  only  record  point-by-point  samples.  This  makes  it  difficult, 
if  not  impossible,  to  collect  spatial  information  from  high-frequency  textures,  i.e.  textures 
that  are  not  solid  color  patches.  In  addition,  animal  coloration  studies  using  spectrometers 
require  getting  the  optical  fiber  very  close  to  the  skin  of  the  animal;  this  is  challenging 
while  studying  freely  behaving  animals  in  their  natural  habitats.  The  use  of  such 
equipment  underwater  further  complicates  the  data  collection  process  due  to  the  rapidly 
changing  light  field  from  the  undulations  of  the  sea  surface,  limited  light  penetration  at 
depth,  particles  suspended  in  water  that  affect  visibility.  Furthermore,  wave  surge, 
currents,  and  practicality  issues  such  as  bulky  watertight  housings  affect  the  speed  and 
maneuverability  of  divers,  and  the  corrosion  of  equipment  from  seawater  and  reduced 
performance  from  most  batteries  due  to  low  operating  temperatures  also  distinguish  such 
marine  studies  from  terrestrial  endeavors. 

Imagers  that  record  continuous  spectra  for  every  pixel  in  an  image,  namely  multi- 
and  hyper-spectral  cameras,  are  becoming  common  in  many  fields  of  science,  and  their 
use  has  been  pioneered  by  (Chiao  et  al.  201 1)  in  the  field  of  animal  coloration.  While  the 
costs  of  such  imagers  are  still  prohibitive,  rapid  developments  in  technology  are  making 
their  deployment  in  the  field  as  fast  as  commercial  off-the-shelf  digital  cameras,  and  they 
will  likely  replace  spectrometers  to  become  the  standard  in  studies  of  coloration  in  the 
next  decade. 

A  non-biological  measure,  SAM,  is  a  rough  estimate  of  color  contrast,  AS 

Little  is  known  about  the  predators  of  A  officinalis  and  their  visual  systems. 

While  retinal  properties  of  most  fish  eyes  are  known,  this  information  does  not 
necessarily  represent  how  they  perceive  scenes;  and  it  most  cases,  it  is  difficult  to  do 
psychophysics  experiments  with  fish  to  develop  better  models  of  their  visual  systems. 
Here,  we  show  that  a  purely  mathematical  spectral  shape  similarity  measure,  SAM,  is 
moderately  correlated  to  the  biological  models  of  color  contrast  we  used.  This  level  of 
correlation  holds  for  di-  and  trichromats,  as  well  as  the  range  of  photoreceptor  ratios  and 
S,  M,  L  peak  wavelengths  we  tested  for.  In  cases  where  there  may  not  be  enough 
information  about  the  visual  system  of  a  particular  observer,  or  a  result  that  can  be 
generalized  across  observers  is  desired,  SAM  could  be  used  to  score  the  similarity  of 
shape  between  two  spectra  (shape  similarity  roughly  corresponds  to  a  similarity  in 
chromaticity).  Traditionally  an  observer-independent  color  difference  metric  D  (Eq.  4.2) 
based  on  hue  and  chroma  is  used  to  estimate  similarity  of  spectral  shape  (Endler  1990). 
We  found  that  this  measure  also  has  a  moderate  degree  of  correlation  to  the  color  contrast 
values  computed  here.  SAM  could  be  computed  alongside  hue  and  chroma  whenever  a 
rough  and  objective  spectral  similarity  score  needs  to  be  calculated.  Eor  our  datasets, 
SAM  and  hue  were  better  correlated  than  SAM  and  chroma;  SAM  and  D  were  weakly 
correlated.  SAM  and  D  individually  correlate  moderately  with  color  contrast  while  they 
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are  weakly  correlated  with  each  other;  this  may  be  because  of  their  different  working 
mechanisms.  The  segment  classification  method  takes  advantage  of  the  opponency  of 
color  channels  while  SAM  is  only  mathematical  measure  of  shape.  These  results  should 
be  interpreted  with  caution  since  they  do  not  relate  to  any  biological  system  and  most 
terms  such  as  luminance,  color,  chromaticity  etc.  are  meaningful  with  respect  to  visual 
systems.  More  specifically,  they  don’t  refer  to  the  visibility  of  colors.  We  expect  a  more 
solid  understanding  of  the  relationship  between  mathematical  and  biological  measures  of 
color  contrast  to  emerge  as  future  studies  adopt  the  calculation  of  a  SAM  score  along 
with  their  biological  analysis. 
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Figure  4.6  Simulation  of  the  spectra  of  the  animal  and  nearhy  substrate  from  Fig.  2&3h,  at  a  depth  of 
10m:  (a)  Normalized  radiance  spectra  of  cuttlefish  components  and  substrates;  red  line  indicates 
normalized  spectra  from  of  a  Spectralon  white  target;  SAM  scores  are  almost  halved  when  compared 
to  Fig.  3b;  luminance  contrast  remains  unchanged  and  color  contrast  has  decreased  significantly,  (b) 
Visualization  of  the  colors  of  cuttlefish  and  substrate  patches  on  a  Maxwell  triangle.  In  the  1  m  case 
the  loci  of  colors  are  relatively  widespread,  but  they  become  almost  coincident  at  10  m.  sRGB 
representation  of  color  appearance  suggests  colors  are  indistinguishable  to  the  human  visual  system 
at  to  meters  depth. 


Assessment  of  spectral  similarity  between  cuttlefish  and  background  in  human  and 
theoretical  fish  predator  color  space 

The  spectral  similarity  between  cuttlefish  and  their  surroundings  was  analyzed 
from  the  perspective  of  the  human  visual  system.  We  calculated  the  tri-stimulus  values  of 
each  spectrum  using  CIE  1931  XYZ  color  space  and  plotted  these  tri-stimulus  values  of 
cuttlefish  body  pattern  components  and  surrounding  substrates  on  an  iso-luminant 


63 


chromaticity  diagram  tuned  for  the  human  visual  system.  This  visual  representation 
provided  a  way  to  assess  the  similarity  of  spectral  properties  between  animal  and 
background.  Cuttlefish  body  components  and  substrates,  were  constrained  to  a  general 
area  around  the  point  x  -0.4  &  y=0.35  near  the  locus  of  the  “white  point”  atx  =  0.33,y  = 
0.33  (Figure  4.5a).  Colors  close  to  this  point  would  appear  gray  under  most  lighting 
conditions.  This  narrow  distribution  implies  that  while  cuttlefish  chromaticity  values  are 
remarkably  close  to  those  of  substrates,  our  study  site  was  not  very  colorful,  at  least  in 
comparison  with  terrestrial  colors  that  humans  are  used  to  (see  black  squares  that  show 
colors  from  a  Macbeth  ColorChecker,  representative  of  colors  that  a  human  might 
observe  terrestrially  on  a  daily  basis). 

In  our  analysis  of  luminance  contrast,  we  only  presented  results  for  trichromats. 
Since  long-  wavelength  receptors  are  thought  to  be  responsible  for  luminance  contrast  in 
fish  (Marshall  et  al.  2003b)  and  we  used  ^rnax  =  570  nm  for  both  the  L  cone  for  the 
hypothetical  trichromatic  predator  and  the  M  cone  for  the  dichromat,  no  differences  were 
expected.  Point-by-point  spectrometer  data  do  not  give  any  spatial  information;  therefore 
luminance  contrast  derived  from  spectral  data  between  cuttlefish  components  alone 
cannot  be  used  to  assess  the  overall  body  pattern  of  a  cuttlefish.  However,  it  is  a  strong 
indicator  of  whether  the  body  pattern  is  uniform  or  non-uniform. 

While  luminance  contrast  remained  the  same  at  depth  for  di-  and  trichromats  in  our 
analysis,  color  contrast  (AS)  showed  some  differences.  The  advantage  of  having  a  third 
visual  pigment  is  best  exemplified  for  the  animal  that  does  not  have  a  good  color  match 
to  the  surrounding  substrate  (Figure  4.2b&Figure  4.3b). 

The  effect  of  water  depth  on  color  matching  of  camouflaged  animals 

Broad-spectrum  sunlight  can  be  available  under  clear  water  at  shallow  depths  on  a 
cloudless  day,  especially  in  kelp  or  coral  reef  habitats.  Under  those  conditions,  colored 
objects  (e.g.,  sand,  rocks,  algae,  coral,  tunicates,  sponges,  etc.)  will  appear  colorful 
(Jerlov  1976;  Chiao  et  al.  2000;  Marshall  et  al.  2003a,  b;  Hochberg  et  al.  2004). 

However,  many  underwater  light  fields  are  not  made  up  of  broad-spectrum  sunlight 
because  light  is  scattered  by  particles  suspended  in  water  (e.g.,  plankton,  sediment,  algae, 
etc.)  and  attenuates  non-uniformly  with  depth  and  wavelength  (Tyler  and  Smith  1970). 
Therefore,  many  marine  habitats  do  no  appear  particularly  colorful,  even  if  the  substrate 
contains  colorful  objects.  In  shallow  coastal  areas,  such  as  our  dive  site,  water  turbidity  is 
key  in  limiting  the  spectral  composition  of  daylight.  At  great  depths,  it  is  thought  that 
camouflage  by  intensity  matching  is  more  effective  than  color  matching,  since  the 
appearance  of  most  objects  become  blue-green  (see  references  in  (Mathger  et  al.  2006).At 
a  depth  of  10  m,  color  contrast  between  the  simulated  animal  spectra  and  substrate 
spectra  was  less  substantial  than  differences  between  actual  animal  and  substrate  spectra 
collected  at  a  depth  of  1  m.  Luminance  contrast,  however,  remained  practically 
unchanged,  as  the  ratios  of  quantum  catches  using  attenuated  spectra  did  not  change 
significantly.  This  result  suggests  that  the  animal  would  have  been  less  distinguishable 
from  its  background  in  terms  of  color  had  the  spectra  been  measured  at  this  depth  (Figure 
4.4),  but  would  have  appeared  to  have  the  same  luminance  contrast  relative  to  the 
surrounding  substrate  despite  attenuated  ambient  light.  Mathger  et  al.  (2008)  performed  a 
similar  simulation  using  laboratory  data  and  showed  that  color  match  differences  adjusted 
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for  a  depth  of  10  m  were  half  as  substantial  as  differenees  measured  in  a  few  eentimeters 
of  water. 

To  visualize  this  attenuation  further,  we  used  Maxwell  triangles  to  represent 
eolors  from  the  visual  system  of  a  hypothetieal  triehromat  predator  (data  only  shown  for 
one  ease,  see  Figure  4.6b).  In  the  ease  of  one  animal  that  appeared  to  have  a  bad  eolor 
mateh  to  its  surroundings  (Figure  4.2b&Figure  4.3b)  at  1  m,  the  euttlefish  and  substrate 
eolors  were  relatively  widely  dispersed  on  the  Maxwell  triangles  (Figure  4.6b).  When  the 
appearance  of  this  animal  was  simulated  at  a  depth  of  10  meters,  the  loci  of  colors  on  the 
Maxwell  triangle  almost  completely  overlapped,  indicating  that  this  animal’s  body 
pattern  would  be  better  camouflaged  in  the  eyes  of  trichomatic  predators  at  a  depth  of  10 
meters. 

Color  matching  in  colorblind  cuttlefish 

Researchers  have  been  puzzled  by  the  color-matching  aspect  of  cuttlefish 
camouflage  because  cuttlefish  are  known  to  be  colorblind  (Brown  and  Brown  1958; 
Marshall  and  Messenger  1996;  Mathger  et  al.  2006).  Mathger  et  al.  2008  suggested  that 
the  spectral  properties  of  S.  officinalis  body  patterns  and  many  natural  objects  are 
generally  similar,  thereby  rendering  color  match  less  difficult.  Our  field  data  confirm 
their  speculation  for  S.  officinalis,  at  least  for  this  particular  study  site.  S.  officinalis 
encounters  a  wide  range  of  habitats  including  temperate  rock  reef  environments 
throughout  the  Mediterranean  and  coral  habitats  off  the  west-central  African  coast. 
Certainly,  some  colors  are  not  in  the  color  repertoire  of  cuttlefish  skin  as  Mathger  et  al. 
(2008)  showed  in  laboratory  studies. 

As  cuttlefish  approach  sexual  maturity,  their  skin  undergoes  a  physiological  and 
morphological  change  where  iridophores  and  leucophores  develop  to  form  White  zebra 
bands  (Hanlon  and  Messenger  1988),  an  important  component  of  body  patterns  used  for 
sexual  signaling.  Although  the  White  zebra  bands  can  be  masked  by  the  overlying 
chromatophores,  they  are  permanent  and  are  often  partially  visible  while  a  mature 
cuttlefish  is  camouflaged.  One  cuttlefish  (Figure  4.2f)  showed  a  weak  zebra  pattern  and 
spectral  measurements  from  individual  light  and  dark  bands  had  low  luminance  and  color 
contrast  relative  to  some  of  the  surrounding  substrates.  This  result  supports  the 
speculation  of  Hanlon  and  Messenger  (1988)  that,  in  addition  to  their  role  in  signaling, 
the  White  zebra  bands  can  contribute  to  camouflage  when  their  bright  contrast  is 
modulated  by  actively  masking  them  to  varying  degrees  by  overlying  chromatophores. 
Overall,  cuttlefish  skin  pattern  components  for  camouflage  closely  resembled  the 
luminance  and  color  of  surrounding  substrates  in  the  eyes  of  hypothetical  di-  and 
trichromatic  fish  predators  we  modeled,  but  the  range  of  colors  found  in  this  particular 
habitat  on  both  cuttlefish  and  substrates  was  narrow.  Nevertheless,  our  light-field  and 
animal/substrate  measurements  corroborate  that  the  spectral  properties  of  chromatophores 
and  natural  objects  are  similar,  thus  facilitating  color  matching  by  cuttlefish.  Moreover, 
cuttlefish  can  neurally  control  the  expression  of  chromatophores  thus  selectively  reveal 
underlying  reflector  cell  types  such  as  leucophores,  which  also  have  some  capability  to 
tone  match  and  perhaps  reflect  ambient  wavelengths  (Messenger  1974).  Despite  apparent 
colorblindness  in  cuttlefish,  the  tone  and  color  matches  between  animal  and  background 
make  cuttlefish  camouflage  superb  in  the  animal  kingdom  (see  selected  images  in 
(Hanlon  et  al.  2009).  It  remains  a  future  challenge  to  discover  how  cephalopods  achieve 
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color  resemblance  to  multiple  backgrounds,  and  to  what  degree  this  is  an  active  vs. 
passive  process.  This  ean  be  approached  in  the  near  future  by  testing  the  eolor  matching 
abilities  of  cuttlefish  in  more  ehromatically  diverse  habitats  (e.g.,  a  coral  reef)  with 
hyper-speetral  imagers  and  fuller  charaeterization  of  the  light  field;  currently,  little  is 
known  about  the  eolor  matehing  abilities  of  cuttlefish  that  live  different  habitats.  Another 
approaeh,  albeit  logistieally  difficult,  would  be  to  transport  cuttlefish  native  to  a 
ehromatically  poor  habitat  to  a  chromatically  rich  habitat.  Many  such  challenges  remain 
in  the  field  of  sensory  eeology,  not  just  of  cephalopods  in  marine  habitats  but  many  taxa 
involved  in  visual  predator/prey  camouflage  interaetions. 


4.6  Conclusion 

Although  cuttlefish  body  pattern  speetra  were  remarkably  similar  to  many  of  the 
natural  substrates  at  the  Aegean  study  site,  the  eolor  spectra  of  those  particular 
background  substrates  and  objects  were  limited  (Fig.  4.4)  and  natural  substrates  that 
cuttlefish  cannot  match  undoubtedly  exist  within  its  geographieal  range.  For  example, 
none  of  the  parts  of  the  animals  in  Figure  4.2c  &  f  closely  matehed  the  dark  green 
Posedonia  in  color  contrast  while  many  of  the  measured  body  pattern  components  elosely 
matehed  tan  and  brown  sand  and  gravel  substrates.  From  our  study,  it  ean  be  eoncluded 
that  there  is  more  to  the  diserimination  of  colors  than  simply  a  spectral  shape  mismateh. 
For  example  CIE  A£'2ooo  (Luo  et  al.  2001;  Sharma  et  al.  2005)  is  a  highly  refined 
measure  of  how  similar  two  colors  appear  to  a  human  observer  and  contains  weighting 
functions  for  lightness,  hue  and  chroma  and  compensation  faetors  for  blue  hue  non¬ 
linearity  that  previous  CIE  eolor  difference  formulations  did  not  have.  The  correlation 
between  SAM  and  A£'2ooo  for  the  data  we  present  here  is  mueh  weaker  (|r|  0.3)  than  the 

correlation  between  SAM  and  A5  (when  this  model  is  used  with  parameters  that  match 
those  of  the  human  visual  system);  this  is  not  beeause  SAM  is  better  matched  to  animal 
visual  systems  but  rather  we  know  more  about  the  human  visual  system  and  thus  have 
derived  a  highly  speeialized  eolor  difference  model.  Regardless  of  how  well  an  animal 
matehes  its  background  chromatically;  its  camouflage  pattern  becomes  more  effeetive  at 
deeper  depths  due  to  the  limited  dynamic  range,  and  uneven  speetral  attenuation 
underwater. 
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Chapter  5 

Image-based  quantification  of  cuttlefish  (S. officinalis) 
camouflage  patterns 

5.1  Introduction 

The  study  of  animal  body  patterning  intertwines  with  many  important  questions  in 
behavioral  and  evolutionary  biology  related  to  mate  selection,  communication,  signaling 
and  camouflage.  For  example,  Hauglund  et  al.  (Hauglund  et  al.  2006)  found  that  the  color 
and  stripes  of  wasp  patterns  are  visual  signals  that  differentially  affect  the  behavior  of 
domestic  chicks  that  hunt  them;  the  yellow  has  an  aversive  effect  on  inexperienced 
predators  while  the  stripes  increase  the  speed  of  avoidance  learning.  While  most  animals 
have  a  pattern  that  undergoes  little  or  no  change  throughout  their  lifetimes,  cuttlefish  are 
able  to  quickly  change  their  appearance  in  response  to  visual  stimuli  (Hanlon  and 
Messenger  1988;  Hanlon  and  Messenger  1996;  Messenger  2001a).  In  this  work,  we  use 
image  processing  and  computer  vision  tools  to  quantify  cuttlefish  camouflage  patterns. 
We  built  an  extensive  database  of  calibrated  photographs  taken  in  the  laboratory  of 
camouflaged  cuttlefish  on  72  different  backgrounds.  Based  on  the  images  in  our  database, 
we  asked  the  following  questions:  (1)  What  is  the  relationship  between  the  levels  of 
expression  of  chromatic  body  components?  (2)  Does  the  space  of  camouflage  patterns 
form  a  continuum,  or  are  there  a  discrete  number  of  clusters?  If  so,  how  many?  (3)  Do 
cuttlefish  express  all  possible  patterns  their  components  are  capable  of  forming,  or  do 
they  only  show  a  subset? 

Animal  patterns  are  used  for  inter-  and  intra-specific  communication,  and  for 
camouflage.  Camouflage  is  an  umbrella  term  that  describes  all  strategies  used  for 
concealment,  including  body  patterning  and  coloration,  mimicking  behavior,  and  motion 
(Stevens  and  Merilaita  2009).  While  there  is  no  widespread  consensus  on  the  definitions 
of  camouflage,  Stevens  and  Merilaita  make  the  distinction  between  function  and  strategy 
to  describe  them:  i.e.  the  function  of  the  evolutionary  adaptation,  such  as  breaking  up 
form;  as  opposed  to  the  strategy,  the  specific  perceptual  processes  targeted,  e.g.  does  the 
pattern  disrupt  normal  edge  detection  mechanisms?  In  this  work,  we  are  most  concerned 
with  the  crypsis  function,  which  aims  to  initially  prevent  detection  and  includes  strategies 
of  background  matching,  distractive  markings,  disruptive  coloration,  self-shadow 
concealment,  obliterative  shading  and  flicker-fusion  camouflage.  Background  matching 
works  through  blending  in  with  the  color,  lightness  or  pattern  of  a  background. 

Distractive  markings  might  stand  out  in  a  way  that  direct  the  attention  of  an  observer 
away  from  the  true  shape  of  the  animal,  where  disruptive  coloration  creates  false  edges 
and  boundaries  to  hinder  the  detection  or  recognition  of  parts  or  the  whole  of  an  animal 
(Stevens  and  Merilaita  2009).  Among  so  many  possibilities,  how  can  we  determine  which 
particular  strategy  an  animal  is  employing?  Stripes  could  disrupt  an  animal’s  outline,  or 
could  form  distractive  markings  (Stevens  and  Merilaita  2009).  How  can  we  decide  what 
their  function  is? 

It  has  often  been  stated  that  our  lack  of  knowledge  of  the  perceptual  mechanisms  of 
the  relevant  observers  hinders  full  understanding  of  camouflage  strategies  (Stevens  and 
Merilaita  2009).  An  additional  obstacle,  which  has  received  little  attention  in  cuttlefish,  is 
the  lack  of  quantification  of  the  limits  to  the  cryptic  coloration  and  patterns  which  can  be 
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produced  by  the  animals.  If  the  animal  does  not  perfeetly  mateh  the  baekground,  is  this 
because  it  cannot,  due  to  limits  in  its  body  patterning  system?  Or  might  the  animal  be 
implementing  an  alternative  eamouflage  strategy,  such  as  disruptive  coloration? 
Camouflage  only  needs  to  deeeive  the  visual  system  of  the  reeeiver  -  if  there  are  limits  to 
the  patterns  that  ean  be  produeed,  does  that  give  us  any  insights  about  the  vastly  different 
visual  systems  of  euttlefish  predators? 

Cuttlefish  skin  exhibits  great  versatility  through  the  display  of  attention-getting 
patterns  for  signaling,  and  highly  eryptie  patterns  for  eamouflage.  This  makes  it  diffieult 
to  infer  what  kind  of  physiologieal  limitations  might  exist  amongst  body  eomponents.  For 
example,  while  the  nervous  eontrol  system  of  the  cuttlefish  allows  for  unilateral 
expression  of  patterns  in  a  conspicuous  appearance,  we  always  observe  a  high  degree  of 
bilateral  symmetry  in  eamouflage  patterns  (Hanlon  and  Messenger  1988;  Langridge 
2006;  Allen  et  al.  2010b);  elearly,  eorrelation  between  right  and  left  side  eomponents  is 
not  a  physiologieal  limitation.  If  there  were  no  limitations  and  all  of  the  cuttlefish’  body 
parts  were  independent  of  each  other,  cuttlefish  eould  theoretieally  express  every  possible 
eombination  of  pattern  eomponents.  Based  on  our  observations  in  the  lab  and  in  the  field, 
some  patterns  are  not  eommon;  a  pattern  with  a  White  Square  darker  than  all  other 
eomponents  has  not  been  reported.  Curiously,  based  on  our  observations,  euttlefish  can 
make  very  dark  Uniform  patterns,  whieh  means  every  component  -  ineluding  the  White 
Square  -  ean  be  made  very  dark.  Why,  then,  does  the  White  Square  never  go  dark  by 
itself?  Understanding  the  limitations  of  eomponents  eould  lend  insight  into  eamouflage 
strategies,  meehanisms,  and  physiology,  as  well  as  into  the  visual  systems  of  their 
predators. 

One  of  the  most  debated  questions  relating  to  cuttlefish  eamouflage  is  whether 
cuttlefish  camouflage  patterns  produee  a  discrete  set  of  patterns,  as  opposed  to  a 
eontinuum  of  patterns.  If  diserete  elasses,  it  beeomes  interesting  to  ask  how  many 
elasses.  A  eontinuum  of  patterns  may  hint  that  euttlefish  might  have  only  a  single 
meehanism  or  strategy  for  produeing  eamouflage  patterns.  On  the  other  hand,  if  the 
animal  sometimes  implements  a  baekground  matching  strategy,  and  sometimes  a  strategy 
of  produeing  distraetive  markings,  then  one  might  expeet  that  the  resulting  patterns 
would  appear  to  form  well-defined  and  separated  pattern  elasses.  In  that  ease,  one  eould 
ask  whether  a  given  elass  serves  a  baekground  matehing  or  a  disruptive  eoloration 
strategy. 

These  questions  ean  be  asked  about  the  overall  luminanee  ehanges  produced  by  the 
larger  seale  body  parts,  and  ean  also  be  asked  of  the  small-to-medium  seale  “mottle” 
patterning.  Is  mottle  patterning  expressed  at  equal  strength  by  eaeh  eomponent  or  are 
some  eomponents  not  wired  to  show  strong  mottle  spots?  Is  mottle  expression  eorrelated 
aeross  the  body  of  the  cuttlefish?  What  are  the  limits  to  mottle  spot  sizes  that  ean  be 
produeed  by  the  cuttlefish  skin? 

In  order  to  explore  the  gamut  of  body  patterns,  we  must  expose  euttlefish  to  a 
maximally  diverse  and  unbiased  colleetion  of  patterns.  Colleeting  images  of  eamouflage 
patterns  is  a  time-eonsuming  proeess,  as  time  to  realize  a  stable  pattern  ranges  from  a 
eouple  of  minutes  to  sometimes  an  hour  or  more,  and  the  number  of  trials  eaeh  animal 
ean  be  exposed  to  in  a  given  day  is  limited  to  prevent  stress  to  the  animal.  To  observe  as 
many  unique  patterns  as  possible,  we  surveyed  the  S.  officinalis  literature  for  baekground 
substrates  used  to  evoke  eamouflage,  and  earned  out  an  extensive  projeet  to  build  a 
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database  of  raw  (linear)  images  of  cuttlefish  camouflaged  on  these  substrates,  as  well  as 
additional  ones  designed  de  novo. 

Consistent  and  controlled  photographic  data  acquisition  is  essential  for  the  success  of 
this  study,  as  it  requires  quantitative  comparison  of  a  large  number  of  body  patterns.  We 
placed  photographic  calibration  targets  in  each  frame  for  dynamic  range  adjustment,  and 
ensured  all  image  intensities  were  scaled  according  to  the  same  standard.  We  call  the 
resulting  set  of  images,  composed  of  5  animals  placed  on  72  different  background 
combinations,  the  Cuttlefish? 2x5  database. 

Using  the  images  in  this  database,  we  measured  body  component  intensity  and 
“mottleness”,  and  derived  a  high-dimensional  feature  matrix.  We  found  that  the 
intensities  of  the  head  and  body  components  are  weakly  negatively  correlated,  indicating 
that  these  parts  may  be  used  as  two  different  elements  of  a  pattern.  In  high-contrast 
patterns,  the  expression  of  the  White  Square  and  the  White  Triangle  are  highly  correlated. 
We  derived  a  map  of  physiological  chromatophore  units  that  control  mottleness,  and 
found  that  for  the  White  Square  and  the  White  Triangle,  both  of  which  are  on  the  axis  of 
bilateral  symmetry,  have  the  least  number  and  total  area  of  dark  spots,  suggesting  that 
one  of  their  functions  might  be  to  create  a  visual  low-frequency  barrier,  disrupting  the 
animal’s  shape.  Our  findings  support  the  qualitative  statement  made  by  Chiao  et  al  (2010) 
that  there  are  two  sizes  of  spots  cuttlefish  can  produce  mottleness  with,  and  that  the  areas 
of  small  light  and  dark  mottle  spots  are  approximately  equal  in  every  pattern  they  are 
expressed.  We  found  that  there  was  no  optimal  number  of  clusters  that  represented  the 
patterns  in  our  database.  Finally,  based  on  principal  components  analysis  of  the  images  in 
our  database,  cuttlefish  show  a  small  subset  of  the  body  patterns  they  could  theoretically 
display,  if  their  components  all  acted  independently. 
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5.2  Background  and  related  work 


Brief  summary  of  animal  pattern  quantification  techniques 

In  this  section,  we  provide  a  brief  overview  of  commonly  used  animal  pattern 
quantification  techniques,  and  then  describe  those  used  for  cuttlefish  patterns  in  detail. 

For  cuttlefish,  the  quantitative  relationship  between  the  levels  of  expression  of  body 
components,  and  their  capabilities,  has  not  previously  been  studied.  Addressing  this 
question  is  important  because  understanding  the  relationships  between  components  is  key 
to  evaluating  the  space  of  all  possible  patterns  that  can  be  shown.  Numerous  studies  have 
investigated,  qualitatively  and  quantitatively,  what  the  total  number  of  camouflage 
patterns  available  to  cuttlefish  might  be  (e.g.,  Hanlon  and  Messenger  1988;  Crook  et  al. 
2002;  Anderson  et  al.  2003).  However,  most  of  these  studies  suffer  from  two  principal 
shortcomings.  First,  it  often  includes  steps  of  scoring  done  by  humans.  While  manual 
scoring  is  quick  and  does  not  require  any  specialized  instruments,  it  is  inherently 
subjective  and  assigned  grades  often  vary  between  observers,  causing  low  repeatability 
(Stevens  2011).  Second,  in  some  cases  patterns  from  different  photographs  are  warped 
onto  a  template  so  certain  features  can  be  compared  locally  when  analyzing  multiple 
images  of  the  same  animal  (Shohet  et  al.  2007;  Anderson  et  al.  2003;  Barbosa  et  al. 
2008b).  This  step  is  error-prone  and  might  produce  blurring  and  aliasing  artifacts 
compromising  data  quality  (Szeliski  2010);  such  problems  have  already  been  reported  in 
the  context  of  cuttlefish  patterning  (Shohet  et  al.  2007;  Anderson  et  al.  2003).  But  more 
importantly,  these  studies  assume  that  well-defined,  discrete  clusters  of  body  patterns 
exist  and  attempt  to  determine  their  number.  Here,  we  will  question  the  validity  of  this 
assumption  and  take  existing  work  further  through  the  use  of  methodologies  that 
investigate  the  structure  of  the  space  of  camouflage  patterns,  while  ensuring  that 
subjective  human  input,  data  acquisition  and  post-processing  artifacts  are  minimized. 

Allen  and  Higham  (Allen  and  Higham  2013)  describe  animal  patterns  as  the  parts  of 
skin,  fur  or  plumage  visible  to  others,  containing  spatial  variations  of  color  and  structure, 
forming  spots,  stripes,  blotches  and  gradients,  often  with  repetitions.  Before  the 
widespread  use  of  computers  and  digital  images,  patterns  and  backgrounds  were  often 
analyzed  by  taking  1  -D  transects  across  (digitized)  images  and  deriving  parameters 
describing  color,  intensity  or  pattern  across  these  transects.  This  process  is  still  frequently 
used  (e.g.,  for  water  snakes  (King  1992);  bowerbird  ornaments  (Endler  and  Day  2006); 
and  for  cuttlefish  component  intensity  (Chiao  et  al.  2009)),  and  is  detailed  in  (Endler 
1984)).  With  the  advent  of  faster  computers,  researchers  have  adopted  frequency  domain 
methods  for  the  analysis  of  2-D  patterns.  These  methods  assess  how  much  information 
patterns  carry  at  various  spatial  scales  and  orientations  e.g.,  tiger  and  zebra  stripes 
(Godfrey  et  al.  1987);  cuttlefish  patterns  (Shohet  et  al.  2006;  Barbosa  et  al.  2008b;  Chiao 
et  al.  2009;  Zylinski  et  al.  2011);  and  avian  eggs  (Stoddard  and  Stevens  2011; 
Spottiswoode  and  Stevens  2012),  and  are  well  suited  to  most  patterns.  The  biggest 
drawback  of  frequency-domain  methods  is  that  spatial  information  is  discarded,  which  is 
important  for  many  research  questions.  Eor  example,  detecting  only  the  energy  carried  by 
mottle  spots  in  cuttlefish  patterns  and  discarding  their  location  prevents  the  understanding 
of  whether  their  expression  is  correlated  among  components,  or  how  it  varies  between 
patterns. 
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Component  Name 

Our 

numbering 

convention 

Hanlon  & 
Messenger 
1988 

convention 

White  arm  triangle 

1 

14 

Anterior  head  bar 

2 

29 

White  head  bar 

3 

13 

Anterior  mantle  bar 

4 

19 

Anterior  transverse 
mantle  line 

5 

17 

White  mantle  bar 

6 

3,  Left 

White  square 

7 

2 

White  mantle  bar 

8 

3,  Right 

Posterior  mantle  bar 

9 

20,  Left 

White  posterior 
triangle 

10 

1 

Posterior  mantle  bar 

11 

20,  Right 

Posterior  transverse 
mantle  line 

12 

18 

Median  mantle  stripe 

13 

22,  Left 

Median  mantle  stripe 

14 

22,  Right 

Figure  5.1  We  number  components  differently  than  the  convention  used  by  Hanlon 
and  Messenger  (Hanlon  and  Messenger  1988),  in  which  Sepia  officinalis  body 
patterns  and  their  components  were  first  described.  We  do  not  consider  some  of  the 
small  chromatic  components  (e.g.,  White  Fin  Spots,  White  Fin  Line,  Pink  Iridophore 
Arm  Stripes  etc.)  and  therefore  a  top-down,  left-to-right  numbering  convention 
utilizing  consequent  numbers  allows  for  more  intuitive  interpretation  of  results.  The 
light  (1,3,6-8,10)  and  dark  (2,4,5,9,11-14)  chromatic  components,  as  described  in 
(Hanlon  and  Messenger  1988)  are  color  coded  here  for  completeness.  Throughout  the 
text,  we  will  prefix  components  with  “C”.  Unless  noted  otherwise,  C6&C8,  C9&C11 
and  C13&C14  will  be  assumed  to  exhibit  bilateral  symmetry  and  only  one  of  each 
pair  will  be  shown  in  results. 


Existing  body  of  work  on  cuttlefish  patterns 

There  has  been  significant  previous  work  describing  cuttlefish  patterns  and 
coloration,  yet  much  of  that  work  has  focused  on  different  questions  such  as  the 
effectiveness  of  camouflage  in  the  eyes  of  predators  (e.g.,  (Akkaynak  et  al.  2013;  Chiao 
et  al.  2011;  Zylinski  et  al.  2009c;  Mathger  et  al.  2008;  Hanlon  et  al.  2013)  and  the 
identification  of  camouflage  strategy  facilitated  by  the  pattern  shown  (e.g.,  (Zylinski  et  al. 
2012;  Buresch  et  al.  2011;  Allen  et  al.  2010b;  Allen  et  al.  2010a;  Barbosa  et  al.  2008b; 
Kelman  et  al.  2008;  Barbosa  et  al.  2008a;  Kelman  et  al.  2007;  Shohet  et  al.  2006; 

Mathger  et  al.  2006;  Chiao  et  al.  2005;  Grable  et  al.  2002),  or  determining  which 
elements  in  a  given  background  influence  final  pattern  expressed  (e.g.,  (Ulmer  et  al. 

2013;  Chiao  et  al.  2013;  Hanlon  et  al.  2013;  Chiao  et  al.  2010;  Lee  et  al.  2010;  Hanlon  et 
al.  2009;  Zylinski  et  al.  2009c,  a;  Barbosa  et  al.  2008b;  Mathger  et  al.  2007;  Barbosa  et 
al.  2007;  Kelman  et  al.  2007;  Chiao  et  al.  2007;  Chiao  et  al.  2005;  Grable  et  al.  2002; 
Chiao  and  Hanlon  2001a,  b;  Marshall  and  Messenger  1996). 

Previous  studies  most  relevant  to  our  research  questions  seek  to  identify  quantifiable 
differences  between  patterns,  and  determine  the  number  of  existing  camouflage  patterns. 
Hanlon  and  colleagues  (Hanlon  and  Messenger  1988)  were  the  first  to  describe  13  overall 
body  patterns,  5  of  which  were  used  for  camouflage,  based  on  visual  observations  in  the 
lab  and  in  the  sea.  The  five  camouflage  patterns  are  generally  sorted  into  three  broad 
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classes  called  Uniform  (U),  Mottle  (M)  and  Disruptive  (D),  allowing  some  variation 
within  eaeh.  These  three  elasses  differ  from  each  other  in  contrast  and  granularity  (spatial 
scale):  Uniform  patterns  have  low  eontrast  and  fine  spatial  seale,  whereas  Mottle  patterns 
have  eoarser  elements  of  moderate  eontrast,  and  Disruptive  patterns  have  large  elements 
of  high  contrast. 

Contrast  quantification 

Chiao  and  colleagues  introduced  a  manual  scoring  system  that  assessed  the  level 
of  expression  of  eaeh  chromatie  body  component  (Chiao  and  Hanlon  2001a,  b),  to 
deseribe  a  body  pattern.  Further  developed  by  Mathger  and  eolleagues  (Mathger  et  al. 
2006),  this  method  is  known  as  disruptive  scoring.  The  term  component  expression  in  the 
context  of  disruptive  scoring  requires  some  clarifieation.  Chromatic  components  are 
divided  into  two  elasses:  dark  and  light  as  defined  in  (Hanlon  and  Messenger  1988).  Dark 
components  on  the  dorsal  side  are  produced  by  expanded  chromatophores  {Figure  5.1, 
components  shown  in  black,  2,4-5, 9,1 1,12-14),  while  light  eomponents  result  from 
retracted  chromatophores  revealing  underlying  leucophores,  translucent  skin  or  the 
euttlebone  {Figure  5.1,  eomponents  shown  in  light  gray,  components  1,  3,  6-8,  10). 
Generally  (somewhat  eonfusingly),  for  disruptive  scoring,  a  light  chromatic  component  is 
considered  expressed  when  it  is  brighter  than  a  threshold,  and  a  dark  component  is 
considered  expressed  when  it  is  darker  than  a  threshold.  Often,  this  threshold  is  chosen  to 
be  the  mean  intensity  of  the  pattern.  When  all  eomponents  are  minimally  expressed 
(score  of  0),  the  body  pattern  is  Uniform;  when  all  are  maximally  expressed  (score  of  3), 
the  body  pattern  is  the  Disruptive  pattern  as  deseribed  in  (Hanlon  and  Messenger  1988). 
Mathger  et  al  used  only  1 1  components  for  scoring,  yielding  a  maximum  score  of  1 1  x  3 
=  33  when  light  components  were  maximally  light,  dark  components  were  maximally 
dark  (Mathger  et  al.  2006).  This  method  was  automated  partially  by  Chiao  et  al.  (Chiao  et 
al.  2009),  by  warping  the  cuttlefish  image  to  a  standard  template  through  user-marked 
correspondenee  points  and  summing  Weber  eontrast  of  components  relative  to  the  image 
mean,  along  transects.  Chiao  and  colleagues  found  that  semi-automatic  disruptive  scores 
predicted  human  assigned  scores  well  (p^  =  0.9683),  suggesting  that  this  method  could 
effectively  replace  human  observers  (Chiao  et  al.  2009). 

Mottleness  quantification 

Uniform,  Mottle  and  Disruptive  pattern  types  differ  in  contrast  and  the  spatial  seale  of 
their  mottle  spots;  and  while  the  disruptive  score  captures  contrast  differences  quite  well, 
it  does  not  describe  differences  in  spatial  seale.  To  quantify  elements  that  vary  in  spatial 
scale,  or  mottleness,  Barbosa  et  al.  introduced  granularity  analysis  (Barbosa  et  al. 

2008b).  This  method  filters  cropped  and  warped  cuttlefish  images  using  6  oetave-wide 
bandpass  filters  and  retains  the  sum  of  the  squared  filter  outputs  (i.e.,  energy)  for  eaeh 
filter.  Two  parameters  then  deseribe  the  pattern:  total  energy  (TE),  which  is  an  indicator 
of  overall  contrast,  and  mean  granularity  (MG),  which  measures  the  average  spatial 
frequency  of  the  body  pattern  (Chiao  et  al.  2009). 
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Other  methodology 

Zylinski  et  al.  (Zylinski  et  al.  2009a)  used  principal  components  analysis  (PCA)  on 
the  manually  graded  expression  of  body  components  (plus,  additional  features  such  as 
mottle  spots)  and  equated  principal  components  to  body  patterns  to  quantify  patterns 
cuttlefish  showed  during  motion  vs'.  when  stationary.  They  found  that  motion  patterns 
were  lower  contrast  than  static  patterns.  Investigating  the  total  number  of  cuttlefish  body 
patterns,  Crook  et  al.  (Crook  et  al.  2002)  used  AutoClass,  (Cheeseman  et  al.  1993)  a 
Bayesian  clustering  algorithm,  and  found  that  12-14  body  patterns  were  most  likely.  They 
used  a  generative  model  approach  to  propose  a  large  set  of  models  that  might  explain 
how  the  body  patterns  were  generated  from  components,  and  then  used  Bayesian 
statistics  to  estimate  the  posterior  probability  of  these  signals  having  been  generated  by  a 
particular  model.  In  addition  to  the  34  chromatic  components  identified  for  S.  ojficinalis 
in  (Hanlon  and  Messenger  1988), 
they  included  six  textural,  eight 
postural  and  six  locomotor 
components  (total  of  54 
components,  N  =  808 
photographs).  The  inputs  to  the 
AutoClass  algorithm  were  the 
human  assigned  scores  of  the 
level  of  expression  of  each  of  the 
54  components.  While  their 
findings  agreed  in  quantity  with 
the  13  patterns  identified  in 
(Hanlon  and  Messenger  1988), 
their  study  made  no  judgment 
about  which  or  how  many  of 
these  might  be  camouflage 
patterns  because  they  analyzed 
behaviors  that  included  feeding, 
foraging  and  social 
communication  in  addition  to 
camouflage.  Using  photographs 
of  cuttlefish  hunting  and  eating 
as  well  as  of  camouflaged 
individuals,  Anderson  and  colleagues  (Anderson  et  al.  2003)  identified  three  temporal 
and  three  spatial  components,  but  the  small  sample  size  (30  cuttlefish  photographs)  limits 
the  effectiveness  of  their  study. 

Next,  we  describe  the  construction  of  the  Cuttlefish? 2x5  database. 


Figure  5.2  Experimental  and  photographic  setup.  The  circular 
cuttlefish  arena  was  placed  in  a  rectangular  tank  supplied  with 
seawater  circulation.  The  setup  was  illuminated  with  a  ring- 
shaped  LED  light,  above  which  the  camera  was  mounted  looking 
down  (not  shown  in  photo).  Black  and  white  calibration  targets 
were  placed  inside  the  rectangular  tank  (shown  here  on  the  left), 
out  of  the  Held  of  view  of  the  animal  in  the  circular  arena  and 
remained  in  the  same  location  throughout  the  duration  of  the 
study. 


5.3  Experimental  Setup 


Animals  Five  adult  (mantle  length:  6. 2-7. 2  cm)  European  cuttlefish  {Sepia  officinalis) 
hatched,  reared  and  maintained  at  the  MBL  Marine  Resources  Center  (Woods  Hole,  MA, 
USA)  were  used  for  the  experiments.  Animals  were  housed  separately  for  the  duration  of 
the  study. 
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Experimental  setup  Each  test  animal  was  placed  in  a  circular  arena,  (23  cm  in  diameter, 
walls  14  cm  high),  inside  a  rectangular  tank  (55x40x15  cm)  with  flow-through  seawater 
(Figure  5.2).  Experiments  ran  over  a  period  of  three  months  (September-December 
2012),  during  which  animals  performed  one  trial  for  every  different  substrate,  and  no 
more  than  two  trials  in  a  single  day  to  minimize  any  stress  on  animals.  To  avoid 
disturbance  to  the  animals  and  control  lighting,  the  experiments  were  conducted  inside  a 
closed  tent  of  black  sheeting  and  the  test  animal  was  observed  using  an  external  video 
monitor.  Each  individual  was  given  a  minimum  of  20  minutes  to  settle  on  the  substrate; 
thereafter,  trials  were  ended  when  the  animal  was  seen  to  have  maintained  a  stable 
position  and  pattern  for  a  period  of  2  minutes. 

Photographic  setup  Black  (Digital  Kolor  Kard)  and  white  (Spectralon;  Eabsphere,  NH) 
photographic  calibration  targets,  which  remained  in  the  same  position  throughout  the 
entire  study,  were  placed  in  the  experimental  setup  in  order  to  allow  for  careful 
calibration  of  the  dynamic  range  in  each  photograph  (Figure  5.2).  These  targets  were  not 
visible  to  the  animals.  To  reduce  shadows  cast  inside  the  arena  and  minimize  lighting 
variation,  the  tank  was  illuminated  by  a  ring  of  FED  lights  (40  cm  diameter),  suspended  5 
cm  above  the  tank.  Photographs  were  taken  every  10s  for  the  duration  of  each  trial  with  a 
Canon  EOS  lOOOD,  flxed  50cm  above  the  tank  and  triggered  remotely  by  a  TimeMachine 
(Mumford  Micro  Systems)  device.  The  tank,  lights  and  camera  remained  in  a  constant 
position  throughout  the  experimental  period. 

Substrates  Substrates  presented  to  the  animals  on  the  floor  and  walls  of  the  arena  were 
chosen  to  capture  the  range  of  backgrounds  previously  tested  in  psychophysics 
experiments  performed  with  cuttleflsh  in  both  our  laboratory  and  others  (Figure  5.3).  To 
elicit  a  variety  of  camouflage  body  patterns,  we  also  designed  novel  substrates  to  test 
specific  characteristics  of  the  cuttlefish  visual  system.  In  total,  we  used  50  substrates, 
natural  and  artiflcial,  two-dimensional  and  textured,  which  were  presented  to  the  animals 
in  random  order.  In  experiments  with  natural  or  artiflcial  substrates,  as  well  as  with 
photographs  of  natural  substrates,  the  walls  were  uniform  grey  50%  (RGB  122),  while 
computer-generated  backgrounds  were  paired  with  both  matching  and  uniform  grey 
walls.  As  a  result,  72  different  background  pattern  combinations  were  tested.  Substrate 
order  was  randomized,  but  remained  the  same  for  each  of  the  flve  individuals. 
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28. 


Figure  5.3  Stimuli  from  existing  studies  (e.g.,  (Hanlon  et  al.  2013;  Chiao  et  al.  2010;  Lee  et  al.  2010; 
Hanlon  et  al.  2009;  Zylinski  et  al.  2009c,  a;  Chiao  et  al.  2009;  Barbosa  et  al.  2008b;  Mathger  et  al. 
2007;  Barbosa  et  al.  2007;  Kelman  et  al.  2007;  Chiao  et  al.  2007;  Chiao  et  al.  2005;  Grable  et  al.  2002; 
Chiao  and  Hanlon  2001a,  b;  Marshall  and  Messenger  1996;  Shohet  et  al.  2007))  form  the  basis  of  the 
images  in  the  Cuttleflsh72xS  database  1  -  Light  brown  sand,  2-  White  sand,  3-  Gravel,  4-Dark  gravel, 
5-Shell  bits  dark,6-Shell  bits  light,  7-Pink  coralline,  8-Gray  shells,  9-Pebbles,  10-Brown  pebbles,  11- 
Black  and  white  sand,  12-Black  and  white  gravel,  13-Black  and  white  pebbles,  14-Uniform  gray  RGB 
122, 15-Uniform  dark  gray,  RGB  61, 16-Black  RGB  0, 17-Checkerboard  4mm,  18-Checkerboard 
5mm,  19-Checkerboard  12mm,  20-Checkerboard  20mm,  21-Scramble  PI  5x5  (1.27mm),  22-Scramble 
PI  5x5  (5.5mm),  23-Tiled  checks  10mm,  24-  Tiled  checks  (5x7mm),  25-Horizontal  stripes,  26-Vertical 
30°  stripes,  27-Vertical  45°  stripes,  28-Frequency  sweep,  29-Sine  wave  1,  30-Sine  wave  2,  31-Sine 
wave  3,  32-Cow  BP72_pt5,  33-Cow  BP36_pt5,  34-Cow  BP18_pt5,  35-Photograph  of  sand,  36- 
Photograph  of  gravel,  37-Photograph  of  gray  shells,  38-Photograph  of  pebbles,  39-Round  black 
beads,  uniform  black  background,  40-Round  black  beads,  black  and  white  spotted  beads,  uniform 
black  background,  41-Round  black  beads,  black  and  white  striped  beads,  uniform  black  background, 
42-Round  black  beads,  white  and  gold  beads,  uniform  black  background,  43-Black  and  white  spotted 
beads,  uniform  black  background,  44-Black  and  white  striped  beads,  uniform  black  background,  45- 
White  and  gold  beads,  uniform  black  background,  46-Black  and  white  large  beads,  uniform  black 
background,  47-Black  and  white  spotted  beads,  uniform  white  background,  48-Black  and  white 
striped  beads,  uniform  white  background,49-White  and  gold  beads,  uniform  white  background,  50- 
Black  and  white  large  beads,  uniform  white  background. 


Image  pre-processing  Camera  raw  images  (/)  were  processed  using  black  (B)  and  white 
(W)  calibration  targets  to  obtain  the  white  balanced  image  as  follows: 

jWB  ^  i  =  R,  G,B  (5.1) 

White  balanced  RGB  images  were  then  converted  to  the  LAB  color  space  (Wyszecki  and 
Stiles  2000),  and  only  the  L  (luminance)  channel  was  retained.  Since  all  images  were 
captured  with  the  same  sensor,  the  analysis  was  done  in  the  camera  color  space  and  no 
color  transformation  was  applied  (see  (Akkaynak  et  al.  2014).  The  intensity  images  were 
then  presented  to  an  experienced  user  (DA)  through  custom-made  software  written  in 
MATLAB  (Mathworks,  Inc.,  Natick  MA)  language,  who  marked  the  orientation  the 
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cuttlefish  and  labeled  the  body  pattern  components.  Component-level  labeling  was  done 
based  on  the  definition  of  chromatic  components  in  (Hanlon  and  Messenger  1988)  but 
components  were  numbered  differently  (Figure  5.1).  To  speed  up  processing  for  certain 
tasks,  images  were  automatieally  registered  based  on  the  interseeting  referenee  strings  in 
the  lower  left  comer  of  each  frame  (Figure  5.2).  With  the  user-made  masks,  average 
intensity  for  each  component  in  the  L  image  was  caleulated,  forming  a  360  x  14 
intensity  matrix  for  all  the  images  in  the  Cuttlefish? 2x5  database. 

Shadow  removal  Even  though  we  designed  our  photographie  setup  to  minimize  shadows 
oast  over  the  arena,  outtlefish  showed  high  preference  for  settling  near  arena  walls.  In 
those  instances,  one  side  of  the  cuttlefish  was  darker  than  the  other  in  the  captured 
photographs  (Figure  5.4a).  We  remedied  this  situation  by  taking  advantage  of  oryptio 
pattern  symmetry  (Langridge  2006),  and  matohing  the  smoothed  pixel  intensity 
histogram  of  the  shadowed  half  to  that  of  the  unshadowed  half.  This  operation  resulted  in 
a  visible  seam  in  the  middle  (Figure  5.3o),  which  we  concealed  by  blending  the  two 
image  halves  at  multiple  sub-bands  using  Laplacian  pyramids  combined  with  spatial 
feathering  (Figure  5.4d)  (Burt  and  Adelson  1983).  While  this  approach  only 
approximates  intensity  values  for  the  shadowed  pixels,  it  maintains  loeal  contrast, 
allowing  us  to  find  local  extrema. 


Pixel  intensity 

before  shadow  removal 


►  pixels 


(a)  (b)  (c)  (d)  (e) 

Figure  5.4  Example  of  shadow  removal  results.  For  each  original  image,  the  histogram  of 
the  shadowed  lateral  half  was  adjusted  to  match  that  of  the  unshadowed  half,  which  was 
repeated  at  four  different  scales,  (a)  Original  image  (h)  User-made  right  and  left-half  masks. 
The  shadow  removal  operation  left  a  visible  seam  between  the  two  halves  (c),  which  was 
blended  through  feathering  (d).  (e)  Intensities  of  pixels  along  a  horizontal  transect  across 
the  image  (red  line)  before  and  after  shadow  removal. 


Next,  we  investigate  our  three  main  research  questions  using  the  imagery  in  the 
Cuttlefish? 2x5  database. 
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5.4  What  is  the  relationship  between  the  levels  of  expression  of 
chromatic  components? 

The  appearance  of  a  cuttlefish’s  body  pattern  is  a  combination  of  three  different 
kinds  of  components:  chromatic,  textural,  postural  and  locomotor;  which  can  be 
recombined  at  any  time  to  make  a  new  pattern  (Hanlon  and  Messenger  1996). 
Components  are  made  of  units  (e.g.,  morphological  organization  of  cells  in  the  skin  into  a 
circular  patch),  which  are  in  turn  composed  of  elements  (e.g.,  chromatophores, 
idirophores,  etc.).  The  physiological  hierarchy  between  the  elements,  units  and 
components  of  a  cephalopod’s  body  pattern  is  well  established  (Packard  and  Hochberg 
1977;  Hanlon  and  Messenger  1988).  Generally  parallel  to  this  physiological  hierarchy  is 
a  neural  hierarchy,  in  which  body  patterns  are  organized  at  higher  levels  in  the  brain  (i.e., 
the  optic  lobes),  and  chromatophores  in  the  lower  motor  centers  (Hanlon  and  Messenger 
1996).  Here,  we  are  interested  in  the  functional  hierarchy  between  chromatic 
components:  Does  the  expression  of  one  component  tend  to  trigger  the  expression  of 
other(s)?  Is  the  appearance  of  component  A  correlated  with  that  of  component  B?  Is 
component  A  only  expressed  above  a  certain  threshold  when  component  B  is?  Identifying 
such  relationships  will  enable  accurate  quantification  of  body  patterns,  which  in  turn  will 
help  answer  more  complex  questions  like  ‘How  many  body  patterns  are  there?”  and  give 
us  insights  into  what  aspects  of  predator  vision  the  expression  of  a  particular  combination 
of  components  might  deceive. 


minimum  intensity  shown  maximum  intensity  shown 


Figure  5.5  Minimum  and  maximum  intensities  shown  by  eaeh  component  in  the  Cuttlefish72x5 
database.  (Note:  we  only  present  results  for  one  of  symmetric  component  pairs  6&8,  9&11  and 
13&14,  because  of  imposed  bilateral  symmetry  during  the  shadow  removal  process.)  All 
components  showed  a  similar  range  of  intensities,  within  the  imposed  dynamic  range  limits  of  0 
and  255.  The  White  Square  (component  7)  has  lighter  minimum  and  maximum  values  compared 
to  other  components.  Dashed  lines  indicate  the  upper  and  lower  values  of  intensities  each 
component  showed;  the  darkest  maximum  intensity  shown  had  a  value  of  206  (on  a  scale  of  0- 
255),  while  the  lightest  was  238.  The  lightest  minimum  intensity  shown  was  14,  and  the  darkest 
was  3  (not  labeled).  Inset:  visualization  of  the  minimum,  maximum  and  mean  grayscale  intensity 
of  each  component. 
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Intensity 

Figure  5  shows  the  minimum  and  maximum  intensities  displayed  by  eaeh  eomponent 
aeross  the  360  images  in  our  database.  Even  though  eomponents  are  historieally 
elassified  as  “light”  and  “dark”,  every  component  expressed  a  similar  range  of  intensities. 
The  White  Square  (C7)  was  the  only  (though  modest)  exception;  its  extrema  were  lighter 
than  those  for  all  other  components.  A  principal  components  analysis  done  on  all  the 
images  in  our  database  reveals  that  the  first  principal  component  of  body  part  intensities 
is  the  average  pattern  luminance,  which  explains  over  87%  of  the  variance.  This  is  not 
surprising,  because  cuttlefish  can  show  the  same  pattern  (e.g.,  a  dark  Uniform  and  a  light 
Uniform)  with  varying  levels  of  intensity  (Buresch  et  al,  under  review).  Thus,  to  observe 
the  other  interesting  aspects  of  component  relationships  more  clearly,  we  subtracted  the 
mean  luminance  from  each  image  to  obtain  zero-mean  image  =  Oint  “  ?  where 

lint  is  the  14-long  pattern  intensity  vector  for  a  given  pattern,  and  /r  is  its  mean  intensity. 
The  distribution  of  zero-mean  intensities  for  each  component  is  given  in  Figure  5.6. 


Figure  5.6  Distribution  of  zero-mean  intensities  per  component;  red  curve  shows  the 
kernei  density  estimate;  verticai  red  iine  shows  x=0.  Whiie  every  component  has 
taken  on  vaiues  brighter  and  darker  than  the  pattern’s  overaii  mean  intensity,  the 
White  Square  (C7)  has  overwheimingiy  taken  tighter  vaiues.  In  other  words,  the 
White  Square  was  aimost  never  expressed  darker  than  the  mean  intensity  of  the 
pattern  (p(C7  <  0)  <  0. 027)  based  on  the  cumuiative  density  function  of  its  kernei 
density  estimate.  For  other  light  components  these  probabilities  were:  p(Cl  <  0)  < 

0. 4854,  p(C3  <  0)  <  0. 1014,  p(C6  <  0)  <  0. 4079,  p(C10  <  0)  <  0. 791;  and  for 
dark  components:  p(C2  >  0)  <  0.  5146,  p(C4  >  0)  >  0. 9730,  p(C5  >  0)  > 

0. 6118,  p(C9  >  0)  <  0. 1749,  p(C12  >  0)  <  0. 1684  and  p(C13  >  0)  <  0. 0632. 
Note  that  the  components  on  the  head  (1-3)  showed  a  wider  range  of  relative 
intensities  compared  to  all  other  components,  which  was  significant  (F  =  0.75-4.7199, 
p  <  0.05). 


To  get  an  understanding  of  component  relationships,  we  first  look  at  pairwise 
correlations  between  their  zero-mean  intensities  (Figure  5.7a).  We  observe  the  strongest 
positive  correlation  between  components  12  &  13,  which  are  the  thin,  linear,  dark 
chromatic  components  on  the  body.  This  indicates  that  when  they  are  expressed,  they 
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have  roughly  the  same  intensity,  perhaps  either  blending  in  as  a  part  of  a  low-eontrast 
pattern,  or  standing  out  as  linear  structures  in  a  high-contrast  pattern.  The  strongest 
negative  correlations  are  observed  between  components  2&10  and  3&9.  These  are 
opposite  polarity  (i.e.,  light/dark)  pairs  and  the  negative  correlation  between  them 
indicates  that  they  are  used  as  contrast-building  elements.  Components  on  the  head  (1-3) 
are  positively  correlated  with  each  other,  but  generally  negatively  correlated,  or 
uncorrelated,  with  the  other  components  on  the  body  (6-13).  The  expression  of  the  head 
and  mantle  components  at  contrasting  mean  intensities  might  suggest  the  simultaneous 
use  of  two  different  strategies;  through  one  body  part  blending  in  with  the  background 
and  other  standing  out,  the  animal’s  shape  might  be  disrupted.  We  have  observed  patterns 
with  head/mantle  component  contrast  in  our  database  (Figure  5.7b),  this  result  may  be 
partially  attributable  to  the  illumination  differences  across  the  experimental  arena  (Figure 
5.7c)  because  cuttlefish  often  positioned  themselves  at  a  right  angle  to  the  arena  walls, 
which  might  have  cast  a  laterally  uneven  shadow  on  them.  Based  on  photographs  alone, 
we  cannot  determine  whether  cuttlefish  actually  detected  these  shadows  and  intentionally 
displayed  contrasting  intensities  between  their  heads/bodies  to  match  the  varying  light 
pattern,  or  whether  the  head  and  mantle  simply  registered  different  pixel  intensities  in  the 
photographs  due  to  shadows. 

Figure  5.7  combines  data  from  all  observed  patterns.  Including  a  large  number  of 
patterns  which  are  nearly  uniform  may  obscure  interesting  relationships  present  in  the 
higher-contrast  patterns.  In  Fig.  8,  we  focus  only  on  high  contrast  patterns.  We  select 
high  contrast  patterns  as  those  in  or  above  the  90**^  percentile  of  for  each 

pattern  where  a  is  the  standard  deviation,  and  i  is  the  index  of  each  of  the  14  components; 
this  yields  a  total  of  36  patterns.  In  this  subset  of  patterns,  components  I&2  and  7&I0 
showed  the  strongest  positive  correlation;  and  C12&C13  remain  moderately  positively 
correlated.  Components  1&2  are  commonly  labeled  as  having  opposite  polarity  (light 
expressed  vs.  dark  expressed,  respectively),  but  they  can  show  intensities  ranging  from 
light  to  dark  (Figure  5.6);  and  amongst  the  high  contrast  patterns  in  our  database,  they 
showed  similar,  not  contrasting  intensities.  A  newly  emerged  high  correlation  between 
C7&10  is  more  interesting  because  while  CIO  showed  both  light  and  dark  intensities 
(Figure  5.6),  C7  overwhelmingly  showed  intensities  lighter  than  the  pattern  mean.  Thus, 
the  high  correlation  between  C7&10  indicates  that  in  high  contrast  patterns,  CIO  is 
exclusively  expressed  at  mean  pattern  intensity  or  lighter,  making  it  a  key  element  for 
contrast.  Component  10  expression  is  also  strongly  negatively  correlated  with  all  head 
components  (1-3),  indicating  that  often,  but  not  always,  a  high-contrast  pattern  has  a  dark 
head.  The  lack  of  very  strong  correlations  amongst  components  indicates  that  there  is  not 
a  single  high-contrast  pattern  and  components  can  combine  in  different  ways  to  make 
many  such  patterns;  four  examples  are  shown  in  Figure  5.8b. 

While  looking  at  component  correlations  is  informative,  it  is  difficult  to  infer  what  the 
relationships  between  more  than  two  components  may  be  from  pairwise  plots.  To  remedy 
that,  we  used  a  dendrogram  to  visualize  the  same  data  from  Figure  5.7&Figure  5.8  in  a 
hierarchical  layout.  In  Figure  5.9a,  we  see  three  main  clusters;  components  on  the  head 
(1-3)  along  with  the  mantle  component  nearest  to  the  head  (4,  cluster  1);  thin,  linear,  dark 
components  on  the  body  (5,  12,  13&14,  cluster2);  and  the  remaining,  larger  components 
on  the  body  (6-11,  cluster3).  Cluster  1  appears  to  be  weakly  negatively  correlated  with 
clusters2&3;  which  are  uncorrelated  with  each  other;  these  relationships  parallel  the 


79 


observations  we  made  for  pairwise  eorrelations  in  Figure  5.7.  The  same  three  elusters 
remain  in  the  high-eontrast  subset  of  patterns  (Figure  5.9b)  with  one  important  differenee 
in  elusterS:  C7  beeomes  less  eorrelated  with  all  the  other  eomponents  in  this  cluster.  This 
is  likely  because  the  White  Square  may  or  may  not  be  fully  expressed  in  a  high  contrast 
pattern  (Figure  5.7b).  Component  10  has  a  positive  correlation  with  its  neighboring 
components  9&1 1;  this  behavior  can  be  seen  in  all  of  the  examples  given  in  Figure  5.7b. 
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Figure  5.7  Here  we  show  the  pairwise  correlations  of  zero-mean  intensities  between  components  for 
all  patterns  in  the  Cuttleflsh72x5  database.  Axes  of  each  plot  have  been  adjusted  to  [-3,3].  (b) 
Examples  of  photographs  with  contrasting  head/body  pattern  intensities,  (c)  Examples  of 
photographs  where  the  recorded  head/body  intensity  difference  is  likely  due  to  shadowing  from  arena 
walls. 
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Figure  5.8  (a)  Pairwise  correlations  of  zero-mean  intensity  between  components  for  36  high- 
contrast  patterns,  that  were  selected  as  those  in  the  90*''  percentile  of  Yii=i  for  each  pattern 

where  a  is  the  standard  deviation,  and  i  is  the  index  of  each  of  the  14  components.  Low  contrast 
patterns  were  selected  as  those  in  the  lO"*  percentile  or  less.  Axes  of  each  plot  have  been  adjusted  to 
[-3,3].  (b)  Examples  of  high  and  (c)  low  contrast  patterns  from  our  database. 


Figure  5.9  We  implemented  hierarchical  clustering  (Duda  et  al.  2012)  for  (a)  all  patterns  and 
(h)  high  contrast  patterns  using  MATLAB  (version  8.2.0.701,  R2013h,  Mathworks,  MA)  and 
used  as  parameters  correlation  distance  and  Ward  linkage.  Left-to-right  placement  of 
components  is  arbitrary.  The  vertical  height  of  each  /e«/ indicates  the  level  of  correlation 
between  the  components  on  that  leaf.  The  horizontal  dashed  line  indicates  the  point  of  no 
correlation,  below  which  correlations  are  positive,  and  above,  negative.Components  9&11, 
6&8  and  13&14  have  perfect  correlation  since  we  enforced  bilateral  symmetry. 


Mottleness 

Up  to  this  point,  we  investigated  the  functional  hierarchy  between  components  only 
considering  their  mean  intensities.  However  cuttlefish  camouflage  patterns  are  often 
mottled',  that  is,  they  contain  small-  and  large-scale  light  and  dark  spots.  These  spots  -are 
expanded  chromatophores,  or  physiological  units  of  chromatophore  patterns  (Packard 
1972)  (Chiao  et  al.  2010).  Chiao  and  colleagues  explain  the  physiological  breakdown  of 
light  and  dark  spots  as  follows: 

“...The  large  scale  component  is  mainly  composed  of four  sets  of  Paired  Mantle 
Spots,  that  surround  the  white  square,  whereas  the  small-scale  mottle  component  is 
represented  by  dark  splotches  of  chromatophore  aggregations,  interspersed  with  light 
patches  throughout  the  mantle  and  arms  (resulting  from  the  retraction  of 
chromatopores).  By  varying  the  expression  strength  and  frequency  of  these  large  and 
small-scale  mottle  components,  cuttlefish  are  able  to  produce  a  variety  of  mottle  patterns. 
The  size  is  limited:  they  only  have  two  sizes,  small  and  large.  ” 

We  used  the  images  in  our  database  to  identify  these  small  and  large  size  mottle 
spots.  After  carefully  registering  the  images  and  warping  them  onto  a  template,  we 
manually  marked  pixels  that  were  consistently  dark.  Even  though  warping  is  potentially 
an  error-prone  operation,  it  was  suitable  in  this  case  because  we  only  used  the  warped 
images  to  obtain  the  map  in  Figure  5.10a,  visually  verifying  the  result  against  the  original 
photographs,  and  did  not  use  the  warped  images  as  inputs  into  further  processing.  We 
were  able  to  obtain  the  map  in  Figure  5.10a  because  the  dark  spots  were  approximately  in 
the  same  locations  in  the  bodies  of  all  five  animals  used  in  our  experiments.  Through 
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Figure  5.10  (a)  The  map  of  dark  mottle  spots,  or  physiological  chromatophore  units  that  control 
the  mottleness  of  patterns,  derived  hy  overlaying  all  images  in  our  database  and  marking  spots 
that  were  consistently  dark.  C5  &  C12  were  omitted  from  this  analysis,  (h)  The  ratio  of  the 
number  of  dark  spots  (#  spots)  per  component  area  (Acomp)  is  approximately  equal  for  all 
components,  except  C7&C10.  (c)  The  ratio  of  the  total  spot  area  within  a  component  (>lspot)  to 
that  component’s  area  (Acomp)  shows  some  variation.  Specifically,  C7&C10  have  the  smallest 
ratio  of  spot  area  per  component  area,  followed  hy  the  components  on  the  head  (Cl-3). 
Components  adjacent  to  C7&C10  have  the  highest  ratio  (C4,  6,  9, 13). 


visual  inspection  this  appears  to  be  true  for  animals  from  completely  different  colonies; 
e.g.,  see  Fig.  2  in  Chiao  et  al.  (2010),  or  Fig.  2  in  Allen  et  al  (2010). 

There  are  a  few  things  to  note  about  Figure  5.10.  The  ratio  of  the  number  of  dark 
spots  to  component  area  is  roughly  equal  for  all  components  other  than  C7&C10,  which 
also  have  the  smallest  ratio  of  total  spot  area  to  component  area  (i.e.,  dark  spot  density). 
These  are  light  components,  which  are  weakly  positively  correlated  (Figure  5.9a),  located 
on  the  axis  of  bilateral  symmetry.  Their  limited  mottleness  capability  might  indicate  that 
one  of  their  functions  might  be  to  provide  a  low  frequency  visual  barrier  between  the 
right  and  left  halves  of  a  pattern,  disrupting  the  body  shape,  or  may  simply  be  facilitating 
better  background  matching  against  substrates  with  linear  (or,  elongated)  elements. 
Components  surrounding  C7&C10,  namely  C4,  C6&C8,  C9&C1 1,  C13&C14,  on  the 
other  hand,  exhibit  the  highest  ratio  of  dark  spot  area-to-component  area,  which  suggests 
that  at  least  for  some  patterns,  the  visual  effect  of  separating  C7&C10  from  the  rest  of  the 
body  might  be  enhanced.  Looking  at  Figure  5.10,  it  is  possible  to  imagine  a  pattern  with  a 
“ring”  of  highly  mottled  components  surrounding  white  and  non-mottled  components  C7 
&  CIO.  But  is  it  possible  for  cuttlefish  to  actually  show  a  pattern  in  which  components 
have  different  levels  of  mottleness?  In  other  words,  is  the  mottleness  of  a  given 
component  correlated  with,  or  independent  of  mottleness  of  other  components?  To 
answer  this  question,  we  built  a  mottleness  detector  (Figure  5.11)  and  quantified 
expression  of  the  mottle  spots  for  each  image  in  our  database,  at  the  component  level. 

Our  detector  is  similar  to  the  granularity  analysis  method  of  Barbosa  et  al.  (2008b)  and 
Chiao  et  al.  (2010)  but  differs  in  that  our  processing  takes  place  in  the  spatial  domain,  and 
retains  the  exact  locations  of  the  found  mottle  spots.  Chiao  and  colleagues  summed  the 
energy  of  patterns  in  4-6  frequency  bands,  representing  different  scales  of  mottleness 
with  a  scalar  at  each  scale.  Their  work  entirely  focused  only  on  mottle  patterns,  and 
therefore  they  used  background  substrates  that  only  evoked  mottle  patterns;  which 
formed  a  subset  of  the  patterns  used  in  our  study  as  our  goal  was  to  generate  as  many 
camouflage  patterns  as  possible.  We  search  for  mottle  spots  at  four  different  spatial 
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Figure  5.11  (a)  Block  diagram  for  detection  of  mottle  features.  Images  are  first  smoothed  using  a 
Gaussian  filter  with  cr  =  5  pixels  (average  size  of  a  cuttlefish  image  is  900  x  400  pixels)  and  then 
decomposed  into  a  four-layer  Gaussian  pyramid.  At  each  scale,  they  are  processed  hy  a  center- 
surround  filter  composed  of  two  concentric  discs  (inner  disc  diameter  varied  from  3  to  9  pixels, 
outer  disc  kept  at  11  pixels).  Filtered  results  are  averaged  over  all  inner  diameter  sizes,  yielding  a 
single  filtered  image  at  each  pyramid  level.  [Is  there  a  non-linearity  before  the  second  filtering?  If 
not  you  are  effectively  running  a  hig,  weird,  linear  filter.]  The  number  of  Gaussian  pyramid  levels 
is  chosen  such  that  the  inner  disk  at  the  coarsest  scale  is  slightly  smaller  than  the  dimensions  of  the 
cuttlefish’s  White  Square;  ensuring  the  detection  of  features  smaller  than  the  White  Square.  As  the 
final  step,  filtered  images  at  each  scale  are  normalized  to  fall  within  the  range  [0,1]  (for  display 
purposes  only),  and  thresholded  to  eliminate  noise  components,  (b)  Examples  of  some  light  and 
dark  spots  found  at  different  scales. 


scales;  the  finest  scale  (Figure  5.1  lb,  scale  1)  was  selected  based  on  the  size  smallest 
dark  spot  we  have  observed,  and  the  coarsest  scale  (Figure  5.1  lb,  scale  4)  was  sized  to  be 
just  smaller  than  the  animal’s  White  Square  (C7).  This  ensures  detection  of  only  light  and 
dark  features  smaller  than  C7  and  prevents  the  expressed  White  Square  from  being 
identified  as  a  large,  light  mottle  spot. 

Using  the  images  in  our  database,  we  first  look  for  evidence  of  the  two  sizes  of 
mottle  spots  hypothesized  by  (Chiao  et  al.  2010).  Our  mottleness  detector  used  four 
spatial  scales  to  search  for  spots  rather  than  assuming  that  the  spots  oceurred  at  partieular 
seales.  We  standardized  the  area  of  each  dark  spot  relative  to  the  aetual  size  of  the 
cuttlefish  image,  and  tested  whether  the  spots  come  from  the  same  distributions.  Through 
visual  inspection,  we  determined  that  the  features  identified  at  scale  1  were  the  same  order 
of  magnitude  of  image  noise  and  excluded  this  scale  from  our  analysis.  We  found  that  the 
area  of  spots  at  soales2&3  were  log-normal  distributed  and  a  two-sample  Kolmogorov- 
Smimov  test  confirmed  they  were  from  the  same  distribution  (p<0.05).  This  distribution, 
in  turn,  was  different  than  that  of  scaled  (Wilcoxon  rank  sum,  p<0.05);  indicating  that 
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there  were  two  scales  of  spots  in  our  database  images.  The  mean  value  of  small  and  large 
spot  areas  differed  roughly  by  an  order  of  magnitude  {1J.2  ~  Ms  =  —5.01,  =  —3.25). 

Next,  we  used 
hierarchical  clustering  to 
investigate  whether  mottle 
spot  expression  was 
correlated  across 
components.  We  used  scale  2 
to  represent  the  small-scale 
mottle  spots  and  scale  4  for 
large  scale.  We  found  that 
certain  components 
consistently  clustered 
together  at  small  and/or  large 
scales,  but  the  correlations 
within  them  were  weak  (0.2 
<p<0.55).  Visualizations  of 
these  correlations  are  shown 
in  Figure  5.12.  Components 
around  the  White  Square 
(C7),  namely  6&8,  9&1 1  and 
13&14  clustered  together  for 
small  and  large  scale  mottle 
spot  expression,  while  C4,  C7 
&  CIO  formed  one  cluster  for 
expressing  small  scale  spots. 

Components  on  the  head  (1- 
3)  expressed  large-scale 
mottle  spots  together,  while 
their  small-scale  expression 
was  clustered  with  the  large- 
scale  expression  of  C4, 

C7&C10.  The  relationship 
between  C7&C10  in  the 
expression  of  mottleness 
parallels  that  of  intensity. 

Finally,  we  found  that 

the  difference  between  the  mean  areas  of  small  dark  and  light  spots  is  on  the  order  of 
10“^,  supporting  the  qualitative  observations  of  Chiao  et  al  2010  that  at  that  scale,  the 
areas  of  light  and  dark  spots  are  roughly  equal  (see  Fig.  2  in  (Chiao  et  al.  2010). 
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Figure  5.12  Visualization  of  clusters  of  small  and  large-scale 
mottle  spot  correlations  across  components,  based  on  hierarchical 
clustering  (with  parameters  correlation  distance  and  Ward 
linkage)  of  component  dark  spot  densities.  Small  and/or  large- 
scale  spots  were  expressed  in  a  weakly  positively  correlated  way 
(0.2  <p<0.55)  in  the  color  coded-clusters.  In  one  case  (lower  right), 
the  small-scale  spots  of  the  head  components  were  correlated  with 
the  large-scale  spots  located  on  the  components  on  the  axis  of 
symmetry  (C4,  7,10). 
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5.5  Do  the  space  of  camouflage  patterns  form  a  continuum,  or  are 
there  a  discrete  number  of  clusters?  If  so,  how  many? 

Previous  attempts  to  answer  these  questions  have  often  used  subjective  judgments  by 
human  observers.  The  human  visual  system  is  a  powerful  pattern  analysis  machine, 
however  a  fundamental  disadvantage  of  determining  pattern  classes  based  on  human 
visual  texture  perception  is  that  inadvertently  those  properties  that  are  more  salient  for  the 
human  visual  system  receive  more  weight,  regardless  of  how  they  may  be  perceived  by 
non-human  visual  systems.  But  more  importantly,  our  words  to  describe  textures  (i.e. 
lexical  texture  space)  are  limited  in  part  because  there  are  more  features  in  textures  than 
those  that  are  important  for  human  visual  texture  perception  (Ravishankar  Rao  and  Lohse 
1996).  For  example,  two  textures  with  similar  statistics  can  be  found  such  that  it  might  be 
very  hard  for  humans  to  tell  them  apart  even  if  the  patterns  in  the  textures  are  quite 
different  (Julesz  1962).  This  discrepancy  between  visual  and  semantic  spaces  arises  from 
the  existence  of  low-level  features  that  computers  can  extract,  but  which  are  not  easily 
perceivable  by  humans.  Thus,  limits  on  human  texture  perception  and  texture  naming 
may  bias  our  assessment  of  the  number  and  type  camouflage  patterns,  if  not  also 
supported  by  objective  methods  of  analysis.  Indeed,  in  their  book  describing  cuttlefish 
behavior,  Hanlon  &  Messenger  noted  that  it  was  difficult  to  name  the  body  patterns 
because  “they  are  dynamic  and  subject  to  immediate  change  and  countless  gradations” 
(Hanlon  and  Messenger  1996). 


(a)  (b) 

Figure  5.13  A  hypothetical  example  with  bivariate  data  showing  well-separated,  discrete 
clusters  in  (a)  and  a  continuum  in  (h). 

To  quantitatively  represent  camouflage  patterns,  we  derived  an  18-dimensional 
feature  matrix  encoding  intensity  and  texture  of  each  pattern.  Four  of  these  dimensions 
carry  information  about  the  texture  properties  (i.e.,  mottleness)  of  the  overall  body 
pattern,  and  the  remaining  14  are  the  zero  mean  intensities  of  each  component.  Due  to 
enforced  bilateral  symmetry,  three  of  these  intensities  are  redundant  (components  6,9  and 
13  are  symmetric  with  8,  1 1  and  14,  respectively),  which  reduces  the  dimensionality  of 
the  feature  matrix  to  15.  Fifteen  dimensions  sufficiently  represent  the  properties  of  each 
pattern,  but  pose  a  visualization  challenge.  In  the  simple  case  of  two  or  three  dimensions, 
we  can  get  intuition  about  the  structure  of  the  data  (i.e.  whether  there  are  distinct  clusters 
or  not)  simply  by  visual  inspection  (Figure  5.13),  but  this  is  impractical  when  data  have 
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more  than  three  dimensions.  Dimensionality  reduetion  teehniques  (e.g.,  prineipal 
eomponents  or  linear  diseriminant  analysis)  ean  be  used  to  projeet  data  onto  fewer 
dimensions-  usually  those  retaining  the  highest  varianee-  making  visualization  easier  and 
eomputing  faster.  However,  the  retained  dimensions  may  not  be  optimal  for 
diseriminating  between  elusters  and  may  obscure  discrete  clusters  present  in  the  data.  To 
avoid  such  manipulation,  we  investigated  the  structure  of  our  data  directly  in  15- 
dimensional  space  using  hierarchical  clustering.  In  general,  hierarchical  clustering 
algorithms  have  complexity  0(n^)  and  are  computationally  prohibitive  for  high¬ 
dimensional  datasets.  However,  for  a  small  number  of  samples  as  in  our  case  (360)  and 
much  smaller  number  of  dimensions  (15),  this  cost  is  easily  manageable  (0.14  seconds  on 
a  2.3  GHz  Intel  Core  i7  processor). 

Figure  5.14a  shows  that  depending  on  how  the  dendrogram  is  partitioned 
(horizontally);  there  might  be  2-14  well-separated  clusters.  But  closer  inspection  of 
patterns  shows  that  none  of  these  clusters  are  necessarily  meaningful  because  there  is 
great  variation  within  each.  We  color-coded  the  branches  of  the  dendrogram  to  match  the 
ordered  thumbnails  in  Figure  5.14b.  Visual  inspection  shows  that  within  a  color-coded 
clade,  patterns  have  high  similarity,  but  these  patterns  differ  from  those  in  other  branches 
in  subtle  or  significant  ways.  However,  it  is  clear  that  if  we  had  chosen  a  lower  threshold 
to  partition  the  dendrogram  horizontally  we  would  have  obtained  fewer  samples  in  each 
cluster,  but  more  agreement  within  clusters. 

What,  then,  is  the  optimal  number  of  clusters?  In  the  field  of  pattern  classification,  it 
is  standard  to  vary  the  number  of  classes  the  data  can  be  partitioned  into,  and  then  assess 
the  stability  of  each  solution.  Here,  we  tested  all  possible  numbers  of  clusters  (ranging 
between  1  and  360),  and  looked  for  a  stable  solution  using  the  Davies-Bouldin  (DB) 
index  (Davies  and  Bouldin  1979).  A  stable  clustering  solution  has  low  within-cluster 
distances,  and  high  between-cluster  distances,  yielding  a  low  DB  value.  Figure  5.14c&d 
shows  that  there  is  no  optimal  number  of  clusters  for  our  data  because  the  DB  value 
keeps  falling  until  every  image  is  in  a  cluster  of  its  own;  i.e.  k  =  360. 

At  this  point  it  is  important  to  note  that  computational  methods  like  hierarchical 
clustering  will  produce  clusters  for  any  set  of  input  data.  Thus,  the  interpretation  of  their 
outputs  is  best  done  based  in  the  context  of  the  research  questions  asked.  For  example, 
are  the  low-level  differences  that  separate  cluster  i  from  cluster)  perceivable  to  a 
particular  visual  system?  If  not,  they  might  safely  be  considered  to  be  a  single  cluster.  For 
example,  a  particular  predator  might  be  highly  sensitive  to  low  spatial  frequency  contrast, 
but  not  as  sensitive  to  contrast  in  high  spatial  frequencies.  This  sensitivity  on  the  part  of 
the  observer  will  affect  the  distance  between  patterns  and  between  clusters.  The  analysis 
above  essentially  applies  for  a  visual  system  as  sensitive  as  our  image  processing 
algorithms.  Rather  than  providing  a  final  answer,  our  methodology  and  results  provide  a 
first-level  basis  upon  which  researchers  can  build. 
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Figure  5.14  (a)  Clustering  of  the  360  images  in  our  database  using  intensity  and  mottleness 
features.  We  found  the  optimal  distance  (Euclidean  distance)  and  linkage  (Ward)  parameters 
using  the  cophenetic  correlation  coefficient,  which  is  a  validation  measure  to  gauge  how 
faithfully  the  chosen  distance  and  linkage  combinations  used  for  dendrograms  represents 
pairwise  distances  between  the  original  data  points  (da  Fona  Costa  and  Cesar  2012).  (b)  Each 
thumbnail  is  ordered  from  left  to  right,  top  to  bottom,  and  color-coded  to  match  its  location  in 
the  dendrogram.  Note  that  mean  image  intensity  was  removed  from  the  data  used  for  clustering, 
while  the  thumbnails  do  show  variations  in  absolute  mean.  This  figure  can  be  seen  in  higher 
resolution  in  the  supplementary  materials,  (c)  A  stable  clustering  solution  should  have  a  low 
Davies-Bouldin  (DB)  index.  Selecting  2  or  14  clusters  has  the  same  DB  value,  and  there  is  no 
global  minimum.  The  lowest  index  is  obtained  when  every  image  is  in  a  cluster  of  its  own 
(k=360),  indicating  that  there  is  no  optimal  number  of  clusters. 
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5.6  Do  cuttlefish  express  all  possible  patterns  their  components 
are  capable  of  forming,  or  do  they  only  show  a  subset?  If  so,  which 
ones? 

Body  components  of  cuttlefish  are  the  building  bloeks  of  body  patterns.  This 
modular  structure  allows  us  to  enumerate  all  possible  permutations  of  eomponents  to 
explore  the  spaee  of  all  (theoretieally  possible)  body  patterns.  How  many  sueh  patterns 
exist?  The  answer  depends  on  how  body  patterns  are  defined.  In  the  Cuttlefish? 2x5 
database  image  intensities  are  eneoded  as  8-bit  values;  that  is,  eaeh  eomponent  is 
represented  using  the  256  intensity  values  between  0  and  255.  If,  for  simplicity,  we  omit 
mottleness  altogether,  and  assume  that  eaeh  of  the  1 1  eomponents  expressed  intensity 
independently,  a  total  of  256^^  =  3.1  x  10^^  patterns  would  be  possible—  a  thousand 
times  more  than  the  ratio  of  atoms  per  moleeules  in  one  mole!  Are  those  patterns  all 
uniquel  For  our  researeh  interests,  we  define  a  unique  pattern  to  be  one  that  has  a  distinct 
spatial  layout  of  elements.  In  other  words,  we  eonsider  two  patterns  to  be  the  same  even 
if  they  vary  in  mean  intensity,  as  long  as  they  have  the  same  design  (i.e.,  the  same  Izm)- 
One  way  to  investigate  if  euttlefish  ean  make  all  the  patterns  their  eomponents  are 
eapable  of  making,  or  just  a  subset,  is  to  look  at  the  prineipal  eomponents  (PC)  of  Izm- 
For  the  images  in  our  database,  the  first  six  PCs  eumulatively  explain  96%  of  the 
varianee.  This  suggests  that  the  observed  patterns  fall  in  a  fairly  low-dimensional 
manifold  within  the  spaee  of  all  possible  patterns;  far  lower,  even,  than  we  would  expeet 
if  the  360  patterns  in  our  database  were  eaeh  unique. 

If  cuttlefish  cannot  show  all  or  most  of  the  patterns  they  are  (theoretically)  capable 
of  making,  whieh  ones  do  they  make?  When  mean  intensity  variations  are  taken  out  and 
only  zero-mean  images  are  eonsidered,  the  number  of  theoretieally  possible  patterns  is 
redueed  signifieantly  by  eonsidering  only  the  set  of  patterns  with  eomponents  at  their 
extreme  values;  effectively,  with  eaeh  eomponent  “On”  or  “Off’.  In  Figure  5.15a,  we 
give  an  example  of  this  for  the  2-dimensional  case;  we  eall  the  resulting  four  patterns 
templates.  The  observed  patterns,  whieh  do  eontain  intensity  variations,  are  plotted  in  the 
spaee  defined  by  these  four  templates;  we  plotted  360  sueh  hypothetieal  patterns  in 
Figure  5.15b  (blaek  dots).  Next,  we’d  like  to  determine  to  which  templates  the  patterns  in 
our  database  are  most  similar.  One  way  to  do  so  is  to  fit  a  statistical  distribution  to  the 
observed  patterns  (for  the  hypothetieal  example  in  Figure  5.15b,  a  one-eomponent 
bivariate  Gaussian),  and  eompute  the  Mahalanobis  distanee  (MD)  from  this  distribution 
to  eaeh  of  the  four  templates.  The  magnitude  of  the  MD  is  an  indieation  of  how  likely  a 
particular  template  might  have  eome  from  that  distribution;  the  smaller  the  MD,  the  more 
likely. 

We  extended  this  approaeh  to  the  images  in  our  database,  obtaining  the  “On”  and 
“Off’  values  of  each  component  using  their  maximum  and  minimum  intensities  from 
Figure  5.5  (adjusted  to  zero-mean),  respectively.  This  yielded  a  total  of  2^^  =  2048 
unique  pattern  templates  (Figure  5.16).  Based  on  their  MD  values  of  the  2048  templates 
to  the  distribution  that  represents  our  data,  we  eonsidered  patterns  in  the  lowest  5 
pereentile  “most  likely”,  and  those  in  the  top  5  pereentile  “most  unlikely”  to  be  produced 
(Figure  5.17),  whieh  coincidentally  yielded  102  patterns  in  both  groups. 
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Figure  5.15  (a)  A  simple  cuttlefish  with  two  components.  If  each  component  can 
only  he  “On”  (light)  or  “Off’  (dark),  there  are  four  possible  patterns;  we  call  these 
templates,  (h)  The  four  templates  form  the  boundaries  of  the  space  in  which 
hypothetical  pattern  observations  (black  dots,  N=360)  fall;  these  patterns  show 
variations  in  intensity  between  [0,1].  In  this  example,  a  bivariate  Gaussian 
distribution  is  fit  to  these  points.  Mahalanobis  distances  between  each  template 
and  this  distribution  are  as  follows:  0  to  (0,0),  6  to  (0,1),  3  to  (0,1),  and  1  to  (1,1). 
Based  on  these  distances,  the  all-dark  template  (0,0)  is  most  likely  and  (0,1)  is  most 
unlikely. 


It  is  useful  to  study  the  patterns  in  Figure  5.17,  but  grasping  the  quantitative 
differences  between  likely  and  unlikely  patterns  and  gauging  why  some  patterns  were 
more  likely  than  others  this  way  is  difficult.  To  make  this  assessment  easier,  in  Figure 
5.18a&b  we  present  visualizations  of  the  PCs  of  the  most  likely  and  unlikely  patterns.  At 
a  glance,  these  visualizations  show  that  the  first  PC  of  the  “most  likely”  set  effectively 
facilitates  high  contrast  between  the  White  Square  and  the  components  on  the  head. 
Patterns  with  such  combinations  account  for  the  biggest  chunk  of  the  variance  in  our 
dataset.  In  complement,  the  first  PC  of  the  “most  unlikely”  set  is  the  lack  of  contrast 
between  the  White  Square  and  the  head  components,  i.e.,  their  correlated  expression.  We 
study  this  PC  in  more  detail  to  see  if  we  can  gain  any  insights  towards  understanding 
what  made  the  patterns  in  Figure  5.17  likely  or  unlikely.  In  Figure  5.18c,  we  re-plot  the 
pairwise  correlations  between  Cl-3  and  C7  from  Figure  5.7a,  and  overlay  on  top  all 
likely  and  unlikely  templates  (red  x’s).  From  this  visualization,  it  becomes  clear  that 
throughout  the  likely  templates,  the  White  Square  is  expressed  almost  entirely  within  its 
limits  observed  in  our  database;  specifically,  we  have  observed  that  Cl-3  can  be 
expressed  dark  when  C7  is  light,  and  Cl-3  can  be  expressed  dark  when  C7  is  dark.  The 
likely  templates  follow  this  pattern.  Throughout  the  unlikely  templates,  we  notice  the 
opposite:  with  the  exception  of  outlier  points,  we  did  not  observe  Cl-3  to  be  expressed 
light  when  C7  is  light;  and  we  did  not  observe  Cl-3  to  be  light  when  C7  is  dark.  The 
“most  unlikely”  templates  capture  these  relationships.  In  fact,  the  relationship  between  all 
other  components  and  C7  in  the  likely  and  unlikely  templates  mirror  the  pattern  shown  in 
Figure  5.18c. 
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We  should  note  here  that  a  pattern  ranked  unlikely  by  our  analysis  does  not  mean 
that  it  is  an  impossible  pattern.  A  pattern  may  have  been  assigned  to  the  unlikely  pile  for 
two  reasons:  it  had  a  very  low  likelihood  of  being  produeed  based  on  the  component 
relationships  in  our  dataset,  or  it  was  not  observed  in  our  database  at  all.  Thus,  it  is 
possible  to  refine  the  space  of  all  possible  patterns  cuttlefish  can  make  by  designing 
background  substrates  that  might  evoke  the  unlikely  patterns,  or  by  considering  images 
from  the  wild  where  cuttlefish  are  subject  to  complex  habitats  and  in  turn  may  show 
different  patterns. 

5.7  Discussion 

In  this  work,  we  quantified  cuttlefish  camouflage  patterns  from  photographs,  ensuring 
that  our  data  acquisition  process  and  methodology  were  maximally  objective.  To  do  so, 
we  minimized  involvement  from  human  observers,  used  calibrated  linear  photographs, 
and  avoided  image-processing  steps  that  might  introduce  artifacts.  Our  goal  was  to  derive 
an  objective  feature  matrix  that  represents  cuttlefish  body  patterns,  which  could  then  be 
used  to  answer  more  complex  questions.  We  focused  on  three  such  questions  in 
particular;  the  relationship  between  the  level  of  expression  of  components,  the  structure 
of  the  camouflage  pattern  space,  and  the  total  number  of  patterns  that  might  be  possible 
for  cuttlefish  to  produce.  We  found  that  C7&C10,  which  are  on  the  axis  of  bilateral 
symmetry,  are  correlated  for  both  intensity  and  mottleness  expression.  In  addition,  these 
light  components  have  the  smallest  number  are  area  density  of  dark  mottle  spots, 
indicating  that  one  of  their  functions  might  be  to  be  expressed  lighter  and  less  mottled 
than  the  surrounding  components,  forming  a  vertical  stripe  between  the  right  and  left 
sides,  visually  disrupting  the  animal’s  body  outline.  This  result  may  be  considered  to  be 
the  first  computational  evidence  towards  determining  whether  cuttlefish  utilize  two 
camouflage  strategies:  background  matching  and  disruptive  coloration. 

Based  our  analyses,  we  propose  two  updates  to  the  terminology  currently  used  to 
describe  cuttlefish  camouflage  patterns.  First,  we  found  that  mottle  features  can  be 
present  on  every  pattern,  suggesting  that  mottleness  is  a  pattern  feature,  rather  than 
Mottle  being  a  unique  pattern  type.  Second,  the  lack  of  strong  correlations  between  all 
“light”  and  “dark”  components  suggests  that  there  is  no  single  high-contrast  pattern,  and 
components  can  combine  in  numerous  ways  to  make  such  patterns.  High  contrast  patterns 
have  historically  been  termed  Disruptive',  yet  as  we  have  shown  in  in  Figure  5.8b,  there 
exist  high-contrast  patterns  in  which  components  like  C7&CI0  that  likely  disrupt  body 
outline  are  not  expressed  as  disruptive  elements.  Even  when  they  are  expressed,  the  term 
Disruptive  more  appropriately  describes  a  subset  of  high-contrast  patterns  rather  than  one 
unique  design. 

The  body  patterns  in  our  data  do  not  clearly  group  into  discrete  and  well- 
separated  clusters.  The  space  of  patterns  observed  in  our  database,  based  on  the  features 
we  have  derived,  appear  to  form  a  continuum.  In  many  ways,  this  result  is  not  surprising 
because  both  intensity  and  mottleness  are  features  that  can  be  expressed  at  varying  levels, 
in  an  intertwined  fashion,  suggesting  that  many  subtly  different  patterns  can  be  made. 
Furthermore,  our  work  only  serves  as  the  first  step  towards  understanding  whether  the 
continuum-like  behavior  of  cuttlefish  patterns  implies  that  cuttlefish  have  only  one 
mechanism  of  producing  camouflage  patterns,  and  therefore  may  (or  may  not)  employ 
only  one  type  of  camouflage  strategy.  Future  work  involves  the  perception  of  patterns 
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through  observer-specific  visual  systems,  which  may  reveal  that  with  respect  to  a 
particular  observer’s  visual  system  some  differences  between  patterns  may  not  be 
discriminable,  and  patterns  may  be  grouped  into  fewer  classes  for  that  observer. 

The  number  of  images  in  our  database  (360)  was  an  order  of  magnitude  smaller  than 
the  number  of  theoretically  possible  patterns  we  hypothesized  cuttlefish  could  show 
(2048).  Since  we  used  as  basis  the  images  in  our  database,  the  properties  of  patterns  that 
were  unlikely  to  be  made  reflect  what  might  be  underrepresented  in  our  database.  A 
future  task  would  be  to  design  substrates  that  will  specifically  evoke  some  of  the  unlikely 
patterns  and/or  use  photographs  of  cuttlefish  patterns  from  the  wild,  which  may  show 
differences  than  those  captured  in  the  lab. 

While  our  work  was  based  entirely  on  images,  it  should  be  noted  that  the  biological 
and  evolutionary  constraints  on  the  design  of  the  nervous  system  and  the  skin  of 
cuttlefish  must  be  considered.  In  any  biological  system  that  can  produce  more  than  a 
single  camouflage  pattern,  there  must  be  a  sensory  system  to  assess  the  background,  a 
brain  to  integrate  and  synthesize  the  information,  and  a  motor  control  system  that 
produces  the  patterns  in  the  skin.  These  are  major  constraints  that  might  help  explain  why 
relatively  few  animals  have  evolved  such  a  capability.  A  key  question  that  has  guided 
much  of  the  biological  investigation  of  rapid  adaptive  camouflage  in  cephalopods  is:  how 
do  they  so  rapidly  assess  and  implement  their  pattern  on  such  diverse  backgrounds  such 
as  coral  reefs  (Hanlon  2007)?  There  must  be  some  parsimonious  solution  involved 
because  there  is  too  much  visual  information  available  to  them,  and  too  many  variations 
of  patterns  that  they  could  produce  with  so  many  chromatophores  and  iridophores 
(millions  of  each  in  a  single  cuttlefish);  controlling  such  a  vast  amount  of  information 
would  take  a  truly  enormous  brain,  presumably  much  larger  than  what  they  actually  have. 
We  hope  that  our  work  will  inspire  more  biological  and  neuro-physiological  research  to 
understand  the  body  component  relationships  and  constraints,  which  could  then  lend 
insights  to  evolution  of  cuttlefish’s  camouflage  patterns. 


5.8  Conclusion 

We  derived  intensity  and  mottleness  features  from  photographs  of  S.  ojjicinalis 
camouflage  patterns  and  used  these  to  answer  three  main  research  questions.  We  used  72 
different  background  combinations  and  5  adult  cuttlefish  to  build  the  Cuttlefish? 2x5 
database  that  contains  360  calibrated  images.  Through  the  analysis  of  these  images,  we 
showed  that  every  body  component  showed  a  wide  range  of  (light-to-dark)  intensities, 
despite  being  labeled  as  “light”  or  “dark”,  except  for  the  White  Square,  which  was 
dominantly  light.  We  found  that  the  components  on  the  head  and  body  were  often 
expressed  in  contrasting  intensities  suggesting  that  the  head  and  the  mantle  may  be  used 
as  two  different  parts  of  a  pattern  employing  different  strategies.  We  also  found  that  CIO 
had  high  correlation  with  C7  in  high-contrast  patterns,  making  those  components 
essential  for  the  creation  of  contrast. 

We  custom  built  a  mottleness  detector  to  quantify  the  expression  of  light  and  dark 
spots  present  in  pattern  at  the  component  level.  Through  the  analysis  of  all  the  images  in 
our  database,  we  made  a  map  of  the  chromatophore  units  that  control  the  expression  of 
dark  spots.  These  spots  appear  to  be  in  the  same  locations  for  all  animals.  Dark  spots 
identified  by  our  mottleness  detector  quantify  the  descriptive  statement  by  Chiao  et  al 
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(2010)  that  the  cuttlefish  skin  only  has  two  sizes  of  mottle;  small  and  large.  We  found 
that  each  component  has  approximately  the  same  number  of  dark  spots  per  its  area,  with 
the  exception  of  C7&C10,  which  have  less.  Similarly,  C7&C10  have  the  least  total  dark 
spot  area  per  component  area  when  compared  to  other  components.  This  relationship 
between  C7&C10  parallels  a  similar  one  in  intensity,  suggesting  that  perhaps  these  light 
components,  located  on  the  axis  of  bilateral  symmetry,  function  as  a  visual  barrier 
between  the  right  and  left  halves  of  a  pattern,  disrupting  the  animal’s  shape,  or  help  blend 
in  better  with  the  elongated  features  of  a  background.  We  also  found  that  the  total  area  of 
small  light  and  dark  spots  are  roughly  equal,  confirming  the  visual  observation  made  by 
Chiao  et  al  (2010). 

Using  an  1 1 -dimensional  feature  vector  composed  of  intensity  and  mottleness,  we 
found  that  the  360  patterns  in  our  database  did  not  discrete,  well-separated  clusters. 
Finally,  based  on  the  inter-component  correlations  we  observed,  we  conclude  that 
cuttlefish  are  likely  to  have  physiological  limitations  that  prevent  them  from  producing 
every  theoretically  possible  body  pattern. 
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Figure  5.16  Binary  cartoons  of  1024  of  the  2048  patterns  cuttlefish  can  theoretically  make 
through  permutation  of  component  expressions.  The  remaining  1024  patterns  are  the  opposite 
polarity  versions  of  those  shown  here. 
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Figure  5.17  The  Mahalanobis  distance  (MD)  between  the  multivariate  distribution  fit  to  Izm 
and  each  of  the  2048  theoretically  possible  patterns  is  a  measure  of  how  likely  these  patterns 
were  observed  in  our  database.  We  defined  those  in  the  S""  percentile,  having  the  smallest 
MD,  as  “most  likely”,  and  those  in  the  95*''  percentile,  having  the  largest  MD,  as  “most 
unlikely”. 
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Figure  5.18  Visualization  of  the  features  each  principal  component  (PC)  controls  in  the  (a)  most 
likely  and  (h)  unlikely  templates;  8  principal  components  retained  89%  and  72%  of  the  variance, 
respectively.  Note  that  these  cartoons  do  not  represent  actual  body  patterns,  and  only  show  the 
positive  and  negative  contrihutions  of  each  principal  component  relative  to  the  average  pattern; 
e.g.,  a  dark  White  Square  does  not  imply  that  the  White  Square  can  he  expressed  that  way.  The 
1st  PC  of  the  “most  likely”  set  facilitates  high  contrast  between  the  C7  and  Cl-3,  and  the 
opposite  in  the  “most  unlikely”  set;  their  correlated  expression.  We  study  this  PC  in  detail  in  (c). 
The  pairwise  correlation  plots  from  Fig.  36a  are  re-plotted  here  for  Cl-3  and  C7  and  the  red  x’s 
show  where  in  this  these  distributions  the  templates  fall.  The  Cl-3  and  C7  relationships  in  the 
likely  templates  parallel  those  observed  in  our  database  well;  while  those  in  the  unlikely  set 
represent  relationships  we  did  not  observe  in  our  database;  such  as  Cl-3  being  expressed  light 
when  C7  is  light,  and  Cl-3  being  light  when  C7  is  dark. 


Chapter  6 


Contributions 

•  Established  a  framework  for  the  use  of  commereial-off-the-shelf  cameras  as 
scientific  data  acquisition  tools  including,  but  not  limited  to,  their  use  in  the  field 
of  animal  coloration  and  camouflage.  This  framework  ensures  consistent  color 
capture  and  can  be  used  on  land  or  underwater. 

•  Established  “scene-specific  color  calibration”  to  maximize  color  capture  accuracy 
of  commercial-off-the-shelf  cameras,  applicable  in  every  field  of  science 
including,  but  not  limited  to,  animal  coloration  and  camouflage.  This  method  can 
be  used  on  land  or  underwater;  its  use  for  underwater  scenes  can  offer  up  to  20- 
fold  decrease  in  color  transformation  errors. 

•  Derived  the  equations  governing  the  freehand  use  of  optical  fibers  that  determine 
the  optimal  measurement  distance  and  angle  from  the  surface  of  a  sample.  These 
equations  can  serve  as  guidelines  for  determining  how  close  a  surveyor  should 
expect  to  get  to  the  surface  being  measured;  e.g.,  skin  of  an  animal,  to  record 
uncontaminated  spectra  from  a  feature  of  known  size.  Alternatively,  for  existing 
measurements,  they  could  quantify  the  degree  of  contamination  if  measurement 
angle  and  distance  were  recorded  as  a  part  of  the  field  study. 

•  Quantified  the  effect  of  contaminated  spectra  on  discriminability  of  colors  in  the 
eyes  of  non-human  animals  for  the  assessment  of  camouflage  effectiveness. 
Showed  that  the  use  of  contaminated  spectra  for  camouflage  assessment  could 
lead  to  (incorrectly)  concluding  that  two  colors  with  minimal  spectral  overlap 
were  indistinguishable  to  a  particular  visual  system. 

•  Quantified,  for  the  first  time,  the  color  match  of  European  cuttlefish  in  the  eyes  of 
its  predators  using  reflectance  and  irradiance  spectra  measured  in  situ.  Showed 
that  cuttlefish  matched  colors  in  their  habitats  closely;  but  the  particular  habitat 
studied  in  the  Aegean  Sea  was  not  colorful. 

•  Built  reflectance  and  irradiance  spectra  databases  characterizing  underwater 
habitats  in  Elrla  (Turkey),  Kona  (Hawaii)  and  Woods  Hole,  (MA).  These 
databases  can  be  used  for  future  color  matching  studiesd,  as  well  as  bio-diversity 
and  ecological  surveys,  and  color-sensitive  underwater  imaging  applications. 

•  Built  the  first  database  (to  be  made  publicly  available)  containing  calibrated  linear 
photographs  of  five  adult  camouflaged  cuttlefish,  taken  against  72  backgrounds;  a 
comprehensive  superset  that  included  ah  backgrounds  used  for  previous  studies  as 
well  as  new  ones.  This  database  also  contains  time-lapse  photographs  of 
cuttlefish’s  settling  behavior  until  a  stable  camouflage  pattern  is  shown. 
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•  Designed  a  “mottleness  deteetor”  for  cuttlefish  body  patterns  (also  applicable  to 
other  animal  patterns):  a  spatial  filter  that  identifies  dark  and  light  spots  at  four 
different  scales.  This  detector  offers  an  advantage  over  existing  ways  of 
quantifying  mottleness  through  capturing  the  spatial  locations  of  dark  and  light 
spots,  as  well  as  their  size  and  shape.  This  enables  the  study  of  more  advanced 
questions,  such  as  which  elements  form  physiological  units  in  the  skin  of  the 
cuttlefish  that  contribute  to  the  consistent  expression  of  mottleness. 

•  Derived  a  high-dimensional  feature  vector  representing  cuttlefish  camouflage 
patterns,  consisting  of  both  intensity  and  mottleness  features.  This  feature  vector, 
which  was  derived  using  objective  and  repeatable  computational  tools,  facilitates 
the  study  of  complex  questions  regarding  cuttlefish  camouflage,  such  as  how 
patterns  differ  from  each  other  or  how  many  patterns  there  might  be. 

•  Quantified  correlations  between  chromatic  body  components  of  European 
cuttlefish  in  terms  of  their  levels  of  expression  of  intensity  and  mottleness.  Most 
recent  physiological  studies  do  not  address  how  cuttlefish  body  components 
function  to  generate  overall  patterns.  Our  data-driven  approach  forms  a  baseline 
for  future  physiological  studies  that  might  investigate  component  relationships 
using  state-of-the-art  techniques. 

•  Employed  pattern  recognition  techniques  to  investigate  the  structure  of  the  high¬ 
dimensional  space  of  cuttlefish  patterns  and  found  they  formed  a  continuum 
rather  than  well-separated  discrete  clusters. 

•  Used  an  image-based  approach  to  perform  a  preliminary  analysis  regarding 
physiological  limitations  cuttlefish  may  have  when  combining  chromatic  body 
components  to  create  overall  camouflage  patterns.  Eound  that  there  appear  to  be 
constraints  to  the  patterns  cuttlefish  can  show,  providing  a  baseline  for  future 
electro-physiological  studies. 
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Chapter  7 


Future  Work 

In  this  thesis,  we  foeused  on  visual  eamouflage  based  on  animal  patterns,  speeifieally 
those  of  euttlefish  (S.  officinalis).  In  biology,  there  is  no  widespread  eonsensus  on  the 
definitions  of  eamouflage  strategies,  whieh  partly  stems  from  our  laek  of  knowledge  of 
the  pereeptual  meehanisms  of  the  relevant  observers,  but  also  from  laek  of  publiely 
available  data  (e.g.,  RGB  or  multi-speetral  photographs  or  point-by-point  speetral 
measurements)  that  is  free  of  instrument  bias.  The  establishment  of  the  Cuttlefish? 2x5 
database  (Chapter  5)  will  faeilitate  deeper  analyses  of  eamouflage  strategies,  as  well  as 
biologieal  questions  related  speeifieally  to  European  euttlefish.  In  partieular,  using  the 
Cuttlefish? 2x5  database,  we  will  be  able  to  analyze  the  time-dependent  settling  behavior 
of  cuttlefish  and  gain  insights  into  how  the  final  camouflage  pattern  may  be  chosen.  This 
behavior  will  also  be  integrated  with  the  statistics  and  spatial  layout  of  the  background 
substrates. 

Through  quantifying  the  dark  mottle  spot  distribution  across  each  body  component, 
and  investigating  their  correlated  expression,  we  found  evidence  that  cuttlefish  might  be 
physiologically  wired  to  utilize  two  camouflage  strategies-  background  matching  and 
disruptive  coloration.  Our  feature  matrix  consisting  of  intensity  and  mottleness  provides  a 
first-level  basis  upon  which  researchers  can  build.  For  example,  we  found  that  using 
these  features  extracted  from  our  dataset,  we  cannot  classify  cuttlefish  patterns  into  well- 
separated  classes.  For  future  work,  a  predator’s  visual  system  can  be  taken  into  account. 
Certain  patterns  that  showed  quantifiable  differences  might  become  indistinguishable 
because  that  predator  may  not  be  sensitive  to  those  features.  Then,  from  the  point  of  view 
of  that  visual  system,  it  would  be  possible  to  classify  patterns  into  discrete  groups.  We 
also  examined  what  the  theoretical  number  of  patterns  cuttlefish  can  produce  might  be, 
and  found  that  cuttlefish  likely  show  a  much  smaller  subset  of  what  their  components  are 
capable  of  showing.  Given  the  theoretical  space  of  all  possible  patterns,  and  those 
patterns  we  actually  observed  in  our  database,  we  found  that  some  patterns  more  likely  to 
be  produced  than  others.  An  important  task  for  future  work  would  be  to  design 
background  substrates  that  might  evoke  some  of  the  patterns  that  were  classified  as 
unlikely. 

As  a  part  of  our  work  investigating  biological  questions  related  to  cuttlefish 
camouflage,  we  have  made  contributions  towards  the  acquisition  of  unbiased  data  using 
conventional  optical  instruments.  In  Chapter  2,  we  presented  a  framework  that  enables 
consistent  and  accurate  color  capture  using  commercial-off-the-shelf  digital  cameras. 
These  cameras  have  applications  in  all  fields  of  science  and  following  this  framework  to 
obtain  properly  calibrated  photographs  can  open  up  new  uses  for  them.  For  example, 
some  applications  of  the  scene-specific  color  calibration  method  described  here  can  be  in 
producing  color-corrected  photomosaics  (underwater,  or  land);  in  mineralogy  where 
minerals  in  soil  could  be  classified  from  photographs;  in  medical  imaging  when  foot  sore, 
or  ulcers  could  be  imaged  by  patients  at  home  and  communicated  to  medical 
professionals  digitally,  or  for  ecological  monitoring  projects  similar  to  the  coral  health 
example  we  have  used  in  Chapter  2. 
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In  Chapter  3,  we  derived  an  equation  that  determines  the  degree  of  speetral 
eontamination  when  the  speetrometer  fiber  is  held  too  far  away  from  the  objeet  being 
measured,  or  at  too  steep  of  an  angle.  This  equation  depends  only  on  the  numerieal 
aperture  and  diameter  of  the  optieal  fiber,  whieh  are  always  known,  therefore  it  ean  be 
used  as  a  guideline  to  eome  up  with  a  measurement  distance  and  angle  estimate  before 
carrying  out  fieldwork  (e.g.,  relative  to  the  size  of  spots  that  will  be  measured  on  an 
animal)  or  any  kind  of  study  that  requires  free-hand  use  of  an  optical  fiber.  Since  this 
equation  only  depends  on  geometrical  optics,  it  is  applicable  for  spectroscopic 
measurements  in  every  field. 

In  Chapter  4,  we  presented  the  first  study  that  used  unbiased  spectrometer 
measurements  taken  in  situ  in  the  Aegean  Sea,  comparing  S.  officinalis  pattern  colors  to 
those  of  its  natural  habitat  (Akkaynak  et  al.  2013).  We  found  that  cuttlefish  showed  a 
good  match  to  colors  in  their  habitats  but  the  range  of  colors  in  the  particular  Aegean 
habitat  we  studied  was  indeed  limited.  For  future  work,  it  would  be  an  interesting 
experiment  to  transport  cuttlefish  from  one  habitat,  e.g.,  the  Aegean,  to  a  more  colorful 
habitat,  such  as  a  coral  reef,  to  examine  whether  their  ability  to  match  color  is  affected 
when  the  gamut  of  their  habitat  colors  changes  noticeably.  In  addition,  based  on 
spectroscopic  measurements  made  from  cuttlefish  skin,  including  microscopic 
measurements  of  chromatophores  previously  done  by  Mathger  et  al.  (2008),  the  gamut  of 
all  colors  cuttlefish  can  (theoretically)  produce  can  be  derived,  establishing  a  basis  of 
comparison  with  the  chromatic  properties  of  natural  habitats. 

Quantification  and  interpretation  of  animal  camouflage  patterns  have  significance  and 
applications  beyond  behavioral  and  evolutionary  biology.  For  example,  understanding 
what  features  of  a  background  an  animal’s  pattern  matches,  successfully  deceiving  the 
observer’s  visual  system,  would  give  us  the  ability  to  replicate  it  with  an  option  to  tune 
the  degree  of  camouflage,  making  the  target  better  or  worse  camouflaged.  For  example, 
such  control  could  be  used  to  magnify  or  reduce  how  well  a  feature  blends  in  with  a 
digital  artwork,  like  a  company  logo  on  a  poster.  Furthermore,  understanding  the 
dimensions  that  control  camouflage  would  give  us  insights  to  reverse  engineer  patterns 
and  use  that  knowledge  expose  hidden  target(s)  in  images.  In  addition  to  defense, 
camouflage -breaking  techniques  have  natural  applications  in  medical  imaging,  where 
cancerous  tissue  is  often  difficult  to  segment  from  healthy  tissue,  e.g.,  in  mammograms, 
endoscopy  or  colonoscopy  images. 
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Appendix 


In  3-space,  ^  defines  a  right  eone  with  its  apex  on  the  origin  0(0, 0,0),  whose 

{R/Hy 

base  is  a  cirele  with  radius  R  in  the  XY  plane,  loeated  at  a  distanee  H  away  from  the  apex 
along  the  z-direetion.  A  plane  that  interseets  this  eone  at  height  Hg,  at  an  angle  a  is 
defined  by  z  =  tan  a  ■  y  +  Hg.  These  two  equations  are  solved  together  to  find  their 
interseetion: 
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Completing  the  square,  the  last  equation  reduees  to 
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From  this,  we  obtain  the  eenter  of  the  ellipse  as  (x^,  yc)  = 
(0,  ’^ith  semi-major  axis  a  =  yjc  (1  —  t^(tana)^)  and  semi-minor  axis 

i/C  “(taritt) 

b  =  Vo.  A  quick  check  shows  that  when  a  =  0  (Xc,yc)  =  (0,0)  and  a  —  b  —  ^/C, 
making  a  eirele  parallel  to  the  base  of  the  cone.'. 
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